[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:19 dist-test-slave-1h5s krb5kdc[18596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:19 dist-test-slave-1h5s krb5kdc[18596](info): set up 2 sockets
Jun 19 06:50:19 dist-test-slave-1h5s krb5kdc[18596](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 06:50:20.021768 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:20 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:20.193084 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39193
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:39193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:20.522302 18609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:20.522670 18609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:20.522843 18609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:20.530129 18609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:20.530308 18609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:20.530423 18609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:20.530527 18609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:20.530615 18609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:20.540560 18609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:39193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:20.542023 18609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:20.543383 18609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:20.556161 18615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:20.556051 18617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:20.556288 18609 server_base.cc:1034] running on GCE node
W20260619 06:50:20.556103 18614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:20.557614 18609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:20.559137 18609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:20.560389 18609 hybrid_clock.cc:648] HybridClock initialized: now 1781851820560347 us; error 46 us; skew 500 ppm
I20260619 06:50:20.560765 18609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:20 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851820, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:20.564023 18609 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:20.565634 18609 webserver.cc:460] Webserver started at http://127.18.39.126:43983/ using document root <none> and password file <none>
I20260619 06:50:20.566375 18609 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:20.566519 18609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:20.566838 18609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:20.570099 18609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "9e920486eb784b91ac983b5b649026be"
format_stamp: "Formatted at 2026-06-19 06:50:20 on dist-test-slave-1h5s"
server_key: "8add09fea01dfd7f7c7cbb07d582cce4"
server_key_iv: "e2059131736634a6c36875f5937dfb62"
server_key_version: "encryptionkey@0"
I20260619 06:50:20.570911 18609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "9e920486eb784b91ac983b5b649026be"
format_stamp: "Formatted at 2026-06-19 06:50:20 on dist-test-slave-1h5s"
server_key: "8add09fea01dfd7f7c7cbb07d582cce4"
server_key_iv: "e2059131736634a6c36875f5937dfb62"
server_key_version: "encryptionkey@0"
I20260619 06:50:20.576292 18609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:20.580003 18624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:20.581578 18609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:20.581782 18609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "9e920486eb784b91ac983b5b649026be"
format_stamp: "Formatted at 2026-06-19 06:50:20 on dist-test-slave-1h5s"
server_key: "8add09fea01dfd7f7c7cbb07d582cce4"
server_key_iv: "e2059131736634a6c36875f5937dfb62"
server_key_version: "encryptionkey@0"
I20260619 06:50:20.582000 18609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:20.600451 18609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:20.604709 18609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:20.605017 18609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:20.624051 18609 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:39193
I20260619 06:50:20.624058 18675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:39193 every 8 connection(s)
I20260619 06:50:20.625592 18609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:20.629439 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 18609
I20260619 06:50:20.629501 18676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:20.629762 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:20.631623 18589 external_mini_cluster.cc:1428] Setting key a0f723d48a37d7555656912dffa8e6ce
I20260619 06:50:20.640908 18676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be: Bootstrap starting.
I20260619 06:50:20.644001 18676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:20.645031 18676 log.cc:824] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:20.648329 18676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be: No bootstrap required, opened a new log
I20260619 06:50:20.655673 18676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } }
I20260619 06:50:20.656073 18676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:20.656195 18676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e920486eb784b91ac983b5b649026be, State: Initialized, Role: FOLLOWER
I20260619 06:50:20.656863 18676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } }
I20260619 06:50:20.657117 18676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:20.657255 18676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:20.657477 18676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 0 FOLLOWER]: Advancing to term 1
Jun 19 06:50:20 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851820, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:20.659848 18676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } }
I20260619 06:50:20.660491 18676 leader_election.cc:302] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e920486eb784b91ac983b5b649026be; no voters:
I20260619 06:50:20.661191 18676 leader_election.cc:288] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:20.661319 18681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:20.662245 18681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [term 1 LEADER]: Becoming Leader. State: Replica: 9e920486eb784b91ac983b5b649026be, State: Running, Role: LEADER
I20260619 06:50:20.662851 18681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } }
I20260619 06:50:20.664636 18676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:20.665975 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e920486eb784b91ac983b5b649026be. Latest consensus state: current_term: 1 leader_uuid: "9e920486eb784b91ac983b5b649026be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } } }
I20260619 06:50:20.666342 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:20.667624 18683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e920486eb784b91ac983b5b649026be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e920486eb784b91ac983b5b649026be" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39193 } } }
I20260619 06:50:20.667985 18683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:20.670257 18686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:20.674578 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:20.635473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41806 (local address 127.18.39.126:39193)
0619 06:50:20.636285 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:20.636303 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:20.637505 (+ 1202us) server_negotiation.cc:407] Connection header received
0619 06:50:20.647239 (+ 9734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:20.647264 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:20.647659 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:20.648127 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:20.651107 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:20.652456 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:20.654244 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:20.654946 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:20.655149 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:20.661818 (+ 6669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:20.661835 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:20.661864 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:20.661936 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:20.666955 (+ 5019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:20.668640 (+ 1685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:20.668652 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:20.668663 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:20.668935 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:20.669714 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:20.669721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:20.669727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:20.670706 (+ 979us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:20.672044 (+ 1338us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:20.672343 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 06:50:20.672758 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":369,"threads_started":1}
I20260619 06:50:20.678463 18686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:20.687433 18686 catalog_manager.cc:1347] Generated new cluster ID: 89d3f02c9256412fbcc59bf3fc8a0949
I20260619 06:50:20.687553 18686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:20.718688 18686 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:20.720242 18686 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:20.735121 18686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e920486eb784b91ac983b5b649026be: Generated new TSK 0
I20260619 06:50:20.736028 18686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:20.778072 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:21.127027 18702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:21.127321 18702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:21.127496 18702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:21.134925 18702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:21.135054 18702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:21.135250 18702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:21.145186 18702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:21.146579 18702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:21.148013 18702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:21.161454 18710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:21.161510 18707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:21.161531 18708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:21.166989 18702 server_base.cc:1034] running on GCE node
I20260619 06:50:21.167712 18702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:21.168707 18702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:21.169941 18702 hybrid_clock.cc:648] HybridClock initialized: now 1781851821169884 us; error 54 us; skew 500 ppm
I20260619 06:50:21.170284 18702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:21 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851821, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:21.173702 18702 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:21.175298 18702 webserver.cc:460] Webserver started at http://127.18.39.65:34391/ using document root <none> and password file <none>
I20260619 06:50:21.176079 18702 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:21.176234 18702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:21.176558 18702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:21.179618 18702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "9ca0660a11d94c06ae5dc09f0db4d146"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "a25dfcce3c2f12322821fbd80f7749fc"
server_key_iv: "40cc19cdd72345f673a3f0c3947bf70e"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.180405 18702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "9ca0660a11d94c06ae5dc09f0db4d146"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "a25dfcce3c2f12322821fbd80f7749fc"
server_key_iv: "40cc19cdd72345f673a3f0c3947bf70e"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.186090 18702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:50:21.189922 18717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.191332 18702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:21.191514 18702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "9ca0660a11d94c06ae5dc09f0db4d146"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "a25dfcce3c2f12322821fbd80f7749fc"
server_key_iv: "40cc19cdd72345f673a3f0c3947bf70e"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.191735 18702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:21.213589 18702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:21.218030 18702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:21.218343 18702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:21.219367 18702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:21.220842 18702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:21.220920 18702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.221043 18702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:21.221108 18702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.254498 18702 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41805
I20260619 06:50:21.254631 18829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41805 every 8 connection(s)
I20260619 06:50:21.255826 18702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:21.257786 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 18702
I20260619 06:50:21.258026 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:21.258755 18589 external_mini_cluster.cc:1428] Setting key 8877d6e416053818020bd1f2255d63d6
Jun 19 06:50:21 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851821, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:50:21.280495 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:21.258740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:58193 (local address 127.18.39.126:39193)
0619 06:50:21.261670 (+ 2930us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:21.261674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:21.261693 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:50:21.263115 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:21.263119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:21.263181 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:21.263273 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:21.265617 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.266446 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.268340 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.269105 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.269257 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:21.273716 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:21.273721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:21.273725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:21.273759 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:21.276692 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:21.277792 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:21.277797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:21.277799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:21.277928 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:21.278465 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:21.278474 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:21.278481 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:21.278804 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:21.278931 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:21.279775 (+ 844us) server_negotiation.cc:299] Negotiation successful
0619 06:50:21.280022 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2828,"thread_start_us":2738,"threads_started":1}
I20260619 06:50:21.281447 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:21.259300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39193 (local address 127.18.39.65:58193)
0619 06:50:21.260292 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:21.260359 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:21.262802 (+ 2443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:21.263565 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:21.263579 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:21.264416 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:21.265380 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:21.265419 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.266608 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.266617 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:21.268179 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:21.268191 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.269322 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.269326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:21.269585 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:21.270375 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:21.270399 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:21.273426 (+ 3027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:21.276874 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:21.276887 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:21.276909 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:21.277623 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:21.278036 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:21.278040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:21.278042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:21.278226 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:21.278917 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:21.278923 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:21.279333 (+ 410us) client_negotiation.cc:769] Sending connection context
0619 06:50:21.279601 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 06:50:21.280202 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":365,"threads_started":1}
I20260619 06:50:21.283138 18830 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39193
I20260619 06:50:21.283514 18830 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:21.284372 18830 heartbeater.cc:511] Master 127.18.39.126:39193 requested a full tablet report, sending...
I20260619 06:50:21.286866 18641 ts_manager.cc:194] Registered new tserver with Master: 9ca0660a11d94c06ae5dc09f0db4d146 (127.18.39.65:41805)
I20260619 06:50:21.289115 18641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:58193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:21.301934 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:21.628990 18836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:21.629271 18836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:21.629449 18836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:21.636863 18836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:21.636977 18836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:21.637163 18836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:21.647351 18836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:21.648733 18836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:21.650133 18836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:21.663564 18841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:21.663489 18842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:21.663441 18844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:21.664878 18836 server_base.cc:1034] running on GCE node
I20260619 06:50:21.665666 18836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:21.666730 18836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:21.667935 18836 hybrid_clock.cc:648] HybridClock initialized: now 1781851821667885 us; error 40 us; skew 500 ppm
I20260619 06:50:21.668272 18836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:21 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851821, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:21.671650 18836 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:21.673377 18836 webserver.cc:460] Webserver started at http://127.18.39.66:46229/ using document root <none> and password file <none>
I20260619 06:50:21.674113 18836 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:21.674260 18836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:21.674583 18836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:21.677659 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "8cf1904ebe3d458f96b2e6efb6953699"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "6de082cf8239afe19164d5079d03b81d"
server_key_iv: "ae57fb4b6e7dab058b0fd7b00e229f02"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.678454 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "8cf1904ebe3d458f96b2e6efb6953699"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "6de082cf8239afe19164d5079d03b81d"
server_key_iv: "ae57fb4b6e7dab058b0fd7b00e229f02"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.683991 18836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:21.687736 18851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.689139 18836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:21.689318 18836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "8cf1904ebe3d458f96b2e6efb6953699"
format_stamp: "Formatted at 2026-06-19 06:50:21 on dist-test-slave-1h5s"
server_key: "6de082cf8239afe19164d5079d03b81d"
server_key_iv: "ae57fb4b6e7dab058b0fd7b00e229f02"
server_key_version: "encryptionkey@0"
I20260619 06:50:21.689577 18836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:21.703179 18836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:21.707350 18836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:21.707665 18836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:21.708645 18836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:21.710011 18836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:21.710112 18836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.710265 18836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:21.710338 18836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:21.742687 18836 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:45663
I20260619 06:50:21.742724 18963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:45663 every 8 connection(s)
I20260619 06:50:21.744022 18836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:21.748090 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 18836
I20260619 06:50:21.748307 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:21.748871 18589 external_mini_cluster.cc:1428] Setting key 47caa8e5a81385cbbb4eff2db7299237
Jun 19 06:50:21 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851821, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:21.767118 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:21.746636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:41353 (local address 127.18.39.126:39193)
0619 06:50:21.746855 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:21.746861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:21.748279 (+ 1418us) server_negotiation.cc:407] Connection header received
0619 06:50:21.750605 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:21.750611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:21.750691 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:21.750826 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:21.753081 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.753890 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.756133 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.756560 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.756698 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:21.760603 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:21.760608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:21.760612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:21.760647 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:21.763402 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:21.764190 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:21.764196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:21.764199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:21.764342 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:21.764811 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:21.764817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:21.764820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:21.765203 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:21.765388 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:21.766482 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0619 06:50:21.766745 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:50:21.768661 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:21.747193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39193 (local address 127.18.39.66:41353)
0619 06:50:21.748026 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:21.748083 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:21.750334 (+ 2251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:21.751103 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:21.751116 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:21.751936 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:21.752867 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:21.752900 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.754049 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.754057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:21.755944 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:21.755958 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:21.756795 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:21.756802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:21.757040 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:21.757798 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:21.757821 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:21.760304 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:21.763575 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:21.763585 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:21.763606 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:21.764059 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:21.764456 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:21.764461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:21.764465 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:21.764694 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:21.765359 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:21.765368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:21.765779 (+ 411us) client_negotiation.cc:769] Sending connection context
0619 06:50:21.766388 (+ 609us) client_negotiation.cc:240] Negotiation successful
0619 06:50:21.767103 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":272,"threads_started":1}
I20260619 06:50:21.770610 18964 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39193
I20260619 06:50:21.770977 18964 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:21.771845 18964 heartbeater.cc:511] Master 127.18.39.126:39193 requested a full tablet report, sending...
I20260619 06:50:21.773579 18641 ts_manager.cc:194] Registered new tserver with Master: 8cf1904ebe3d458f96b2e6efb6953699 (127.18.39.66:45663)
I20260619 06:50:21.774466 18641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:41353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:21.790194 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:22.109737 18969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:22.110102 18969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:22.110285 18969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:22.118793 18969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:22.118966 18969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:22.119143 18969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:22.129325 18969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39193
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:22.130774 18969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:22.132156 18969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:22.145495 18977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:22.145679 18974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:22.145702 18975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:22.146636 18969 server_base.cc:1034] running on GCE node
I20260619 06:50:22.147297 18969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:22.148324 18969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:22.149529 18969 hybrid_clock.cc:648] HybridClock initialized: now 1781851822149490 us; error 46 us; skew 500 ppm
I20260619 06:50:22.149869 18969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:22.153128 18969 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:22.154742 18969 webserver.cc:460] Webserver started at http://127.18.39.67:45003/ using document root <none> and password file <none>
I20260619 06:50:22.155534 18969 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:22.155687 18969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:22.156023 18969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:22.159067 18969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "ea6d3abd39674d579eea8471aa8f0178"
format_stamp: "Formatted at 2026-06-19 06:50:22 on dist-test-slave-1h5s"
server_key: "53557170060cc3770fbdf900ec59e156"
server_key_iv: "2b1b2d2595a8f79dac9045501f451b29"
server_key_version: "encryptionkey@0"
I20260619 06:50:22.159843 18969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "ea6d3abd39674d579eea8471aa8f0178"
format_stamp: "Formatted at 2026-06-19 06:50:22 on dist-test-slave-1h5s"
server_key: "53557170060cc3770fbdf900ec59e156"
server_key_iv: "2b1b2d2595a8f79dac9045501f451b29"
server_key_version: "encryptionkey@0"
I20260619 06:50:22.165299 18969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:22.168980 18984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:22.170625 18969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:50:22.170855 18969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "ea6d3abd39674d579eea8471aa8f0178"
format_stamp: "Formatted at 2026-06-19 06:50:22 on dist-test-slave-1h5s"
server_key: "53557170060cc3770fbdf900ec59e156"
server_key_iv: "2b1b2d2595a8f79dac9045501f451b29"
server_key_version: "encryptionkey@0"
I20260619 06:50:22.171099 18969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:22.195396 18969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:22.199914 18969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:22.200219 18969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:22.201203 18969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:22.202566 18969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:22.202672 18969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:22.202837 18969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:22.202911 18969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:22.236846 18969 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:40663
I20260619 06:50:22.236892 19096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:40663 every 8 connection(s)
I20260619 06:50:22.238174 18969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:22.247016 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 18969
I20260619 06:50:22.247264 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:22.247720 18589 external_mini_cluster.cc:1428] Setting key 797f5b5a2c26e95d2597d32ac673cb7c
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:22.258539 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.240748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:58387 (local address 127.18.39.126:39193)
0619 06:50:22.240939 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.240945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.242066 (+ 1121us) server_negotiation.cc:407] Connection header received
0619 06:50:22.243791 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.243795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.243875 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.243970 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.245423 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.246183 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.247738 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.248348 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.248487 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.252400 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.252405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.252408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.252452 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.255153 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.256021 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.256025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.256027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.256148 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.256593 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.256596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.256598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.256859 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.256965 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.257899 (+ 934us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.258209 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:50:22.259505 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.241184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39193 (local address 127.18.39.67:58387)
0619 06:50:22.241907 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:22.241941 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:22.243540 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:22.244161 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:22.244200 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:22.244678 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:22.245213 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:22.245229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.246304 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.246308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:22.247604 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:22.247615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.248574 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.248582 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:22.248860 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.249622 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:22.249651 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:22.252222 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:22.255294 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:22.255303 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:22.255320 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:22.255912 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:22.256254 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:22.256259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:22.256263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:22.256489 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:22.256984 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:22.256992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:22.257403 (+ 411us) client_negotiation.cc:769] Sending connection context
0619 06:50:22.257700 (+ 297us) client_negotiation.cc:240] Negotiation successful
0619 06:50:22.258196 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":327,"threads_started":1}
I20260619 06:50:22.261015 19097 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39193
I20260619 06:50:22.261317 19097 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:22.262041 19097 heartbeater.cc:511] Master 127.18.39.126:39193 requested a full tablet report, sending...
I20260619 06:50:22.263635 18641 ts_manager.cc:194] Registered new tserver with Master: ea6d3abd39674d579eea8471aa8f0178 (127.18.39.67:40663)
I20260619 06:50:22.264276 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:22.264783 18641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:58387
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260619 06:50:22.292579 18830 heartbeater.cc:503] Master 127.18.39.126:39193 was elected leader, sending a full tablet report...
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:22.309701 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.294443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40492 (local address 127.18.39.65:41805)
0619 06:50:22.295178 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.295185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.295273 (+ 88us) server_negotiation.cc:407] Connection header received
0619 06:50:22.295472 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.295480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.295732 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.295900 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.297421 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.298419 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.299616 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.300148 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.300242 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.304180 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.304190 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.304205 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.304238 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.307106 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.307726 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.307734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.307736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.307864 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.308257 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.308261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.308262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.308584 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.308795 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.309046 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.309230 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":423,"threads_started":1}
W20260619 06:50:22.310943 18745 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40492
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:22.353873 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.339454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.18.39.65:41805)
0619 06:50:22.339698 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.339705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.340074 (+ 369us) server_negotiation.cc:407] Connection header received
0619 06:50:22.340322 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.340327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.340423 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.340539 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.342052 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.342789 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.344054 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.344491 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.344641 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.348659 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.348664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.348667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.348702 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.351159 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.351775 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.351779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.351781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.351889 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.352186 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.352189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.352191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.352422 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.352666 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.353140 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.353431 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:50:22.355690 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 18702
I20260619 06:50:22.369632 19104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.356891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40514 (local address 127.18.39.65:41805)
0619 06:50:22.357121 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.357126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.358115 (+ 989us) server_negotiation.cc:407] Connection header received
0619 06:50:22.358175 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.358180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.358274 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.358367 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.359922 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.360842 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.362153 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.362550 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.362656 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.364817 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.364822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.364825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.364856 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.367395 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.367993 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.367997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.367999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.368131 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.368439 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.368442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.368444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.368698 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.368869 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.369115 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.369269 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:50:22.370297 18825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:22.453492 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 18702
I20260619 06:50:22.469148 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 18836
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:22.485397 19111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.470031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.18.39.66:45663)
0619 06:50:22.470555 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.470562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.470586 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:50:22.470644 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.470651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.470928 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.471180 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.472358 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.473406 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.474686 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.475261 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.475406 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.479409 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.479421 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.479436 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.479479 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.482477 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.483164 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.483171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.483173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.483291 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.483656 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.483660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.483662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.484014 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.484285 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.484548 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.484851 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
I20260619 06:50:22.487191 18959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:22.570017 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 18836
I20260619 06:50:22.585716 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 18969
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[18596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:22.602126 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:22.586763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.18.39.67:40663)
0619 06:50:22.587369 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:22.587376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:22.587405 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:50:22.587461 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:22.587467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:22.587694 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:22.587816 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:22.589008 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.590110 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.591365 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:22.591929 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:22.592026 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:22.596321 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:22.596357 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:22.596373 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:22.596417 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:22.599344 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.600113 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.600121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.600123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.600263 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:22.600590 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:22.600594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:22.600596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:22.600936 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:22.601176 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:22.601503 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 06:50:22.601747 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":327,"threads_started":1}
I20260619 06:50:22.603520 19092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:22.693281 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 18969
I20260619 06:50:22.708750 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 18609
I20260619 06:50:22.710021 18671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:22.813652 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 18609
2026-06-19T06:50:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3891 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[19118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[19118](info): set up 2 sockets
Jun 19 06:50:22 dist-test-slave-1h5s krb5kdc[19118](info): commencing operation
krb5kdc: starting...
W20260619 06:50:23.893404 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.004s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:24 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:24.078962 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38763
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:38763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:24.403360 19131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:24.403666 19131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:24.403838 19131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:24.411451 19131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:24.411584 19131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:24.411686 19131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:24.411785 19131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:24.411846 19131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:24.421970 19131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:38763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:24.423399 19131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:24.424786 19131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:24.436862 19137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:24.436924 19136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:24.436902 19139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:24.438283 19131 server_base.cc:1034] running on GCE node
I20260619 06:50:24.439180 19131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:24.440682 19131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:24.441898 19131 hybrid_clock.cc:648] HybridClock initialized: now 1781851824441834 us; error 62 us; skew 500 ppm
I20260619 06:50:24.442279 19131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:24 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851824, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:24.445678 19131 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:24.447225 19131 webserver.cc:460] Webserver started at http://127.18.39.126:42569/ using document root <none> and password file <none>
I20260619 06:50:24.447966 19131 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:24.448120 19131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:24.448450 19131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:24.451838 19131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "b738a92b8dfd4adf86af01db111fd051"
format_stamp: "Formatted at 2026-06-19 06:50:24 on dist-test-slave-1h5s"
server_key: "c43498a0ffee28f969fafcfb4301a37c"
server_key_iv: "fd509531f6308d1e20a93077a2680c27"
server_key_version: "encryptionkey@0"
I20260619 06:50:24.452672 19131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "b738a92b8dfd4adf86af01db111fd051"
format_stamp: "Formatted at 2026-06-19 06:50:24 on dist-test-slave-1h5s"
server_key: "c43498a0ffee28f969fafcfb4301a37c"
server_key_iv: "fd509531f6308d1e20a93077a2680c27"
server_key_version: "encryptionkey@0"
I20260619 06:50:24.458262 19131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:50:24.462057 19146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:24.463524 19131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:24.463761 19131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "b738a92b8dfd4adf86af01db111fd051"
format_stamp: "Formatted at 2026-06-19 06:50:24 on dist-test-slave-1h5s"
server_key: "c43498a0ffee28f969fafcfb4301a37c"
server_key_iv: "fd509531f6308d1e20a93077a2680c27"
server_key_version: "encryptionkey@0"
I20260619 06:50:24.463985 19131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:24.477734 19131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:24.481892 19131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:24.482234 19131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:24.501117 19131 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:38763
I20260619 06:50:24.501139 19197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:38763 every 8 connection(s)
I20260619 06:50:24.502549 19131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:24.504769 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19131
I20260619 06:50:24.504992 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:24.505726 18589 external_mini_cluster.cc:1428] Setting key ee1eb28ad5c402d343d0d6d1692b8956
I20260619 06:50:24.506429 19198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:24.519165 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051: Bootstrap starting.
Jun 19 06:50:24 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:24.522636 19198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:24.523733 19198 log.cc:824] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:24.526736 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051: No bootstrap required, opened a new log
I20260619 06:50:24.528039 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:24.507732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51502 (local address 127.18.39.126:38763)
0619 06:50:24.508476 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:24.508487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:24.508526 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:50:24.510126 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:24.510147 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:24.510548 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:24.510947 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:24.512423 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:24.514242 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:24.515545 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:24.516258 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:24.516470 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:24.520462 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:24.520470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:24.520485 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:24.520527 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:24.523729 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:24.524527 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:24.524535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:24.524541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:24.524717 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:24.525102 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:24.525108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:24.525111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:24.525725 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:24.526023 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:24.526538 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 06:50:24.526971 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":327,"threads_started":1}
I20260619 06:50:24.535419 19198 raft_consensus.cc:348] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } }
I20260619 06:50:24.535908 19198 raft_consensus.cc:374] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:24.536029 19198 raft_consensus.cc:729] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b738a92b8dfd4adf86af01db111fd051, State: Initialized, Role: FOLLOWER
I20260619 06:50:24.536706 19198 consensus_queue.cc:260] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } }
I20260619 06:50:24.536981 19198 raft_consensus.cc:388] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:24.537122 19198 raft_consensus.cc:482] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:24.537386 19198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:24.539767 19198 raft_consensus.cc:504] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } }
I20260619 06:50:24.540325 19198 leader_election.cc:302] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b738a92b8dfd4adf86af01db111fd051; no voters:
I20260619 06:50:24.540930 19198 leader_election.cc:288] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:24.541076 19203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:24.542613 19203 raft_consensus.cc:686] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [term 1 LEADER]: Becoming Leader. State: Replica: b738a92b8dfd4adf86af01db111fd051, State: Running, Role: LEADER
I20260619 06:50:24.543195 19198 sys_catalog.cc:564] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:24.543134 19203 consensus_queue.cc:237] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } }
I20260619 06:50:24.546316 19204 sys_catalog.cc:455] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b738a92b8dfd4adf86af01db111fd051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } } }
I20260619 06:50:24.546325 19205 sys_catalog.cc:455] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b738a92b8dfd4adf86af01db111fd051. Latest consensus state: current_term: 1 leader_uuid: "b738a92b8dfd4adf86af01db111fd051" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b738a92b8dfd4adf86af01db111fd051" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38763 } } }
I20260619 06:50:24.546813 19205 sys_catalog.cc:458] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:24.546810 19204 sys_catalog.cc:458] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:24.550925 19212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:24.556214 19212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:24.565219 19212 catalog_manager.cc:1347] Generated new cluster ID: 96c0e5569d164c928d24941e359f17a5
I20260619 06:50:24.565377 19212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:24.590662 19212 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:24.592038 19212 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:24.605113 19212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b738a92b8dfd4adf86af01db111fd051: Generated new TSK 0
I20260619 06:50:24.606132 19212 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:24.652906 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:24.998085 19224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:24.998407 19224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:24.998508 19224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:25.005982 19224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:25.006112 19224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:25.006240 19224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:25.016327 19224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:25.017700 19224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:25.019001 19224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:25.032670 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:25.032706 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:25.032598 19232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:25.033686 19224 server_base.cc:1034] running on GCE node
I20260619 06:50:25.034350 19224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:25.035269 19224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:25.036495 19224 hybrid_clock.cc:648] HybridClock initialized: now 1781851825036454 us; error 54 us; skew 500 ppm
I20260619 06:50:25.036809 19224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:25 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851825, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:25.039950 19224 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:25.041515 19224 webserver.cc:460] Webserver started at http://127.18.39.65:36563/ using document root <none> and password file <none>
I20260619 06:50:25.042187 19224 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:25.042325 19224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:25.042636 19224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:25.045735 19224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "708df65d49814b3fa0ca8d230437d56d"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "8c77270dd6792cf8c4bff8f5662f5f69"
server_key_iv: "869efc18133e4e1ca03ea24dd740a259"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.046496 19224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "708df65d49814b3fa0ca8d230437d56d"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "8c77270dd6792cf8c4bff8f5662f5f69"
server_key_iv: "869efc18133e4e1ca03ea24dd740a259"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.051771 19224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:50:25.055510 19239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:25.057040 19224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:25.057314 19224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "708df65d49814b3fa0ca8d230437d56d"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "8c77270dd6792cf8c4bff8f5662f5f69"
server_key_iv: "869efc18133e4e1ca03ea24dd740a259"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.057636 19224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:25.075150 19224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:25.079658 19224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:25.079965 19224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:25.080940 19224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:25.082255 19224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:25.082361 19224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:25.082513 19224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:25.082589 19224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:25.114665 19224 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:34855
I20260619 06:50:25.114707 19351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:34855 every 8 connection(s)
I20260619 06:50:25.115921 19224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:25.123060 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19224
I20260619 06:50:25.123281 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:25.123962 18589 external_mini_cluster.cc:1428] Setting key a65d0d27fc5306d2ee95d2df4c057543
Jun 19 06:50:25 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851825, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:25.139011 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:25.118375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:54267 (local address 127.18.39.126:38763)
0619 06:50:25.118831 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:25.118836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:25.119823 (+ 987us) server_negotiation.cc:407] Connection header received
0619 06:50:25.121748 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:25.121754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:25.121852 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:25.121975 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:25.123680 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.124454 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.126569 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.126952 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.127049 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:25.131547 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:25.131557 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:25.131565 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:25.131612 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:25.135326 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:25.136162 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:25.136171 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:25.136177 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:25.136330 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:25.136829 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:25.136837 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:25.136852 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:25.137242 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:25.137464 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:25.138216 (+ 752us) server_negotiation.cc:299] Negotiation successful
0619 06:50:25.138501 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":241,"threads_started":1}
I20260619 06:50:25.139667 19355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:25.118976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38763 (local address 127.18.39.65:54267)
0619 06:50:25.119660 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:25.119697 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:25.121470 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:25.122229 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:25.122238 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:25.122771 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:25.123499 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:25.123530 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.124644 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.124648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:25.126423 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:25.126440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.127193 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.127204 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:25.127557 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:25.128328 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:25.128365 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:25.131289 (+ 2924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:25.135494 (+ 4205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:25.135503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:25.135519 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:25.135976 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:25.136467 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:25.136471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:25.136473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:25.136692 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:25.137430 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:25.137437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:25.137773 (+ 336us) client_negotiation.cc:769] Sending connection context
0619 06:50:25.138033 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 06:50:25.138484 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":278,"threads_started":1}
I20260619 06:50:25.141292 19352 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38763
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:50:25.141692 19352 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:25.142424 19352 heartbeater.cc:511] Master 127.18.39.126:38763 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:50:25.144737 19163 ts_manager.cc:194] Registered new tserver with Master: 708df65d49814b3fa0ca8d230437d56d (127.18.39.65:34855)
I20260619 06:50:25.146966 19163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:54267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:25.164538 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:25.483624 19358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:25.483990 19358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:25.484164 19358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:25.491590 19358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:25.491766 19358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:25.491956 19358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:25.503470 19358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:25.504804 19358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:25.506135 19358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:25.521621 19363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:25.521718 19364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:25.521694 19366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:25.522418 19358 server_base.cc:1034] running on GCE node
I20260619 06:50:25.523064 19358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:25.524034 19358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:25.525305 19358 hybrid_clock.cc:648] HybridClock initialized: now 1781851825525224 us; error 74 us; skew 500 ppm
I20260619 06:50:25.525719 19358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:25 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851825, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:25.528986 19358 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:25.530733 19358 webserver.cc:460] Webserver started at http://127.18.39.66:45719/ using document root <none> and password file <none>
I20260619 06:50:25.531450 19358 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:25.531595 19358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:25.531911 19358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:25.534946 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "97c958c4a3f5466a9b0f1130633f9573"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "1d92a8c7695a8c32efc49bf4bff8b2fb"
server_key_iv: "ff972c1be98b2ab5104768041638d951"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.535756 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "97c958c4a3f5466a9b0f1130633f9573"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "1d92a8c7695a8c32efc49bf4bff8b2fb"
server_key_iv: "ff972c1be98b2ab5104768041638d951"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.541198 19358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:50:25.544696 19373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:25.546149 19358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:25.546339 19358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "97c958c4a3f5466a9b0f1130633f9573"
format_stamp: "Formatted at 2026-06-19 06:50:25 on dist-test-slave-1h5s"
server_key: "1d92a8c7695a8c32efc49bf4bff8b2fb"
server_key_iv: "ff972c1be98b2ab5104768041638d951"
server_key_version: "encryptionkey@0"
I20260619 06:50:25.546558 19358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:25.565814 19358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:25.570452 19358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:25.570744 19358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:25.571735 19358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:25.573163 19358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:25.573238 19358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:25.573401 19358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:25.573462 19358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 06:50:25.605314 19358 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:46845
I20260619 06:50:25.605379 19485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:46845 every 8 connection(s)
I20260619 06:50:25.606597 19358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:25.611549 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19358
I20260619 06:50:25.611773 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:25.612233 18589 external_mini_cluster.cc:1428] Setting key 37b882ed4370a618c5eeb1de95d298d1
Jun 19 06:50:25 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851825, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:25.626869 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:25.609176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:40693 (local address 127.18.39.126:38763)
0619 06:50:25.609434 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:25.609439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:25.610609 (+ 1170us) server_negotiation.cc:407] Connection header received
0619 06:50:25.612597 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:25.612603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:25.612699 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:25.612810 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:25.614655 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.615353 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.616926 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.617440 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.617576 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:25.621040 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:25.621045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:25.621048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:25.621080 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:25.623720 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:25.624489 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:25.624495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:25.624499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:25.624627 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:25.625059 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:25.625066 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:25.625070 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:25.625408 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:25.625561 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:25.626271 (+ 710us) server_negotiation.cc:299] Negotiation successful
0619 06:50:25.626523 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:25.628038 19488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:25.609764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38763 (local address 127.18.39.66:40693)
0619 06:50:25.610443 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:25.610480 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:25.612311 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:25.613056 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:25.613064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:25.613832 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:25.614484 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:25.614503 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.615523 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.615527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:25.616764 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:25.616775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:25.617671 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:25.617678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:25.617913 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:25.618621 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:25.618648 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:25.620766 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:25.623884 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:25.623894 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:25.623913 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:25.624357 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:25.624738 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:25.624744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:25.624748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:25.624948 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:25.625533 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:25.625542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:25.625909 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 06:50:25.626186 (+ 277us) client_negotiation.cc:240] Negotiation successful
0619 06:50:25.626728 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":267,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:50:25.629613 19486 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38763
I20260619 06:50:25.629927 19486 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:25.630694 19486 heartbeater.cc:511] Master 127.18.39.126:38763 requested a full tablet report, sending...
I20260619 06:50:25.632282 19163 ts_manager.cc:194] Registered new tserver with Master: 97c958c4a3f5466a9b0f1130633f9573 (127.18.39.66:46845)
I20260619 06:50:25.633569 19163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:40693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:25.649778 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:25.970307 19491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:25.970604 19491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:25.970705 19491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:25.978097 19491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:25.978226 19491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:25.978343 19491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:25.988417 19491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38763
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:25.989707 19491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:25.991024 19491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:26.004235 19499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:26.004340 19496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:26.004356 19497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:26.005203 19491 server_base.cc:1034] running on GCE node
I20260619 06:50:26.005872 19491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:26.006865 19491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:26.008149 19491 hybrid_clock.cc:648] HybridClock initialized: now 1781851826008083 us; error 52 us; skew 500 ppm
I20260619 06:50:26.008530 19491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:26.011823 19491 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:26.013299 19491 webserver.cc:460] Webserver started at http://127.18.39.67:34589/ using document root <none> and password file <none>
I20260619 06:50:26.014057 19491 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:26.014199 19491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:26.014513 19491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:26.017680 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "d09a5ceb11d344f8b985e2a23c0e7733"
format_stamp: "Formatted at 2026-06-19 06:50:26 on dist-test-slave-1h5s"
server_key: "25e571db1298b1c3ddf7e2bdd1a4dcf6"
server_key_iv: "c83fe7df31a18be2fcd1f0c281e97df9"
server_key_version: "encryptionkey@0"
I20260619 06:50:26.018481 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "d09a5ceb11d344f8b985e2a23c0e7733"
format_stamp: "Formatted at 2026-06-19 06:50:26 on dist-test-slave-1h5s"
server_key: "25e571db1298b1c3ddf7e2bdd1a4dcf6"
server_key_iv: "c83fe7df31a18be2fcd1f0c281e97df9"
server_key_version: "encryptionkey@0"
I20260619 06:50:26.023696 19491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:50:26.027334 19506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:26.028795 19491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:26.028986 19491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "d09a5ceb11d344f8b985e2a23c0e7733"
format_stamp: "Formatted at 2026-06-19 06:50:26 on dist-test-slave-1h5s"
server_key: "25e571db1298b1c3ddf7e2bdd1a4dcf6"
server_key_iv: "c83fe7df31a18be2fcd1f0c281e97df9"
server_key_version: "encryptionkey@0"
I20260619 06:50:26.029196 19491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:26.040843 19491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:26.045212 19491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:26.045567 19491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:26.046563 19491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:26.047897 19491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:26.048002 19491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:26.048147 19491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:26.048218 19491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:26.082019 19491 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:45591
I20260619 06:50:26.082067 19618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:45591 every 8 connection(s)
I20260619 06:50:26.083315 19491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:26.086426 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19491
I20260619 06:50:26.086659 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:26.087229 18589 external_mini_cluster.cc:1428] Setting key 0fcf5bf138b29be9f7ddc897fb8ef6dc
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:26.103861 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.085783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:52253 (local address 127.18.39.126:38763)
0619 06:50:26.085955 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.085960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.087224 (+ 1264us) server_negotiation.cc:407] Connection header received
0619 06:50:26.089328 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.089359 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.089458 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.089568 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.091222 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.091955 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.093580 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.093981 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.094109 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.097540 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.097546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.097549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.097583 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.100269 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.101147 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.101154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.101158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.101304 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.102039 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.102043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.102045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.102322 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.102479 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.103278 (+ 799us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.103499 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:50:26.104869 19621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.086223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38763 (local address 127.18.39.67:52253)
0619 06:50:26.087032 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.087084 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.089050 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.089813 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.089823 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.090460 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:26.091037 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.091053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.092120 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.092124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.093430 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.093447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.094213 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.094220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.094467 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.095116 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:26.095138 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:26.097264 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:26.100434 (+ 3170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:26.100446 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:26.100466 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:26.101022 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:26.101454 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:26.101459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:26.101463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:26.101653 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:26.102455 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:26.102463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:26.102858 (+ 395us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.103179 (+ 321us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.103747 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":311,"threads_started":1}
I20260619 06:50:26.106196 19619 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38763
I20260619 06:50:26.106492 19619 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:26.107136 19619 heartbeater.cc:511] Master 127.18.39.126:38763 requested a full tablet report, sending...
I20260619 06:50:26.108494 19163 ts_manager.cc:194] Registered new tserver with Master: d09a5ceb11d344f8b985e2a23c0e7733 (127.18.39.67:45591)
I20260619 06:50:26.109234 19163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:52253
I20260619 06:50:26.115257 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:26.135818 19354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.122725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.18.39.126:38763)
0619 06:50:26.122938 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.122946 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.123333 (+ 387us) server_negotiation.cc:407] Connection header received
0619 06:50:26.123572 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.123577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.123654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.123762 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.125198 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.126008 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.127308 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.127724 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.127874 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.130372 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.130377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.130380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.130413 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.133040 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.133782 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.133789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.133792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.133930 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.134330 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.134336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.134339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.134661 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.134914 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.135267 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.135491 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:50:26.142251 19163 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:26.149417 19163 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:26.150389 19352 heartbeater.cc:503] Master 127.18.39.126:38763 was elected leader, sending a full tablet report...
I20260619 06:50:26.175252 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.162469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:46845 (local address 127.0.0.1:43638)
0619 06:50:26.165814 (+ 3345us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.165852 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.166070 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.167193 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.167202 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.167238 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:26.167635 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.167649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.169711 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.169716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.172794 (+ 3078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.172811 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.174145 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.174150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.174330 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.174385 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.174532 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.174728 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3146,"mutex_wait_us":696,"thread_start_us":3651,"threads_started":1}
I20260619 06:50:26.175897 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.162707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.18.39.66:46845)
0619 06:50:26.163348 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.163355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.165965 (+ 2610us) server_negotiation.cc:407] Connection header received
0619 06:50:26.166424 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.166435 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.166703 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.166884 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:26.167848 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.169532 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.172942 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.173951 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.174114 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.174335 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.175412 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.175607 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":388,"threads_started":1}
I20260619 06:50:26.176213 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.163388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42118 (local address 127.18.39.67:45591)
0619 06:50:26.164082 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.164087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.166526 (+ 2439us) server_negotiation.cc:407] Connection header received
0619 06:50:26.166811 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.166820 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.167086 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.167238 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:26.168345 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.169877 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.172808 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.174289 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.174427 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.174664 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.175312 (+ 648us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.175866 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":419,"threads_started":1}
I20260619 06:50:26.176772 19631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.162629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:45591 (local address 127.0.0.1:42118)
0619 06:50:26.166419 (+ 3790us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.166438 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.166625 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.167582 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.167588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.167620 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:26.168075 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.168138 (+ 63us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.170056 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.170061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.172588 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.172603 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.174818 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.174824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.175076 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.175118 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.176176 (+ 1058us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.176447 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3635,"thread_start_us":4308,"threads_started":1}
I20260619 06:50:26.179790 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.166503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:34855 (local address 127.0.0.1:55162)
0619 06:50:26.170936 (+ 4433us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.170954 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.171071 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.172143 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.172147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.172166 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:26.172518 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.172529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.174862 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.174866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.176890 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.176911 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.179066 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.179073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.179239 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.179271 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.179373 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.179495 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4305,"thread_start_us":4127,"threads_started":1}
I20260619 06:50:26.180179 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.167188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.18.39.65:34855)
0619 06:50:26.171271 (+ 4083us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.171280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.171310 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:50:26.171420 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.171428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.171694 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.171946 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:26.172697 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.174702 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.177095 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.178556 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.178691 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.178917 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.179692 (+ 775us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.179865 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3894,"thread_start_us":4430,"threads_started":1}
I20260619 06:50:26.180119 19554 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0 (DEFAULT_TABLE table=test-table [id=135a98495ac14ed691ecf462ca6f51c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:26.180128 19421 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0 (DEFAULT_TABLE table=test-table [id=135a98495ac14ed691ecf462ca6f51c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:26.181579 19421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:26.181610 19554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:26.183979 19287 tablet_service.cc:1467] Processing CreateTablet for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0 (DEFAULT_TABLE table=test-table [id=135a98495ac14ed691ecf462ca6f51c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:26.185499 19287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:26.193053 19637 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Bootstrap starting.
I20260619 06:50:26.196067 19637 tablet_bootstrap.cc:654] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:26.196960 19638 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Bootstrap starting.
I20260619 06:50:26.198592 19637 log.cc:824] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:26.201023 19638 tablet_bootstrap.cc:654] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:26.201200 19640 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Bootstrap starting.
I20260619 06:50:26.202191 19637 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: No bootstrap required, opened a new log
I20260619 06:50:26.202303 19638 log.cc:824] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:26.202456 19637 ts_tablet_manager.cc:1397] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260619 06:50:26.204650 19640 tablet_bootstrap.cc:654] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:26.205359 19638 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: No bootstrap required, opened a new log
I20260619 06:50:26.205607 19638 ts_tablet_manager.cc:1397] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260619 06:50:26.205832 19640 log.cc:824] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:26.208875 19640 tablet_bootstrap.cc:492] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: No bootstrap required, opened a new log
I20260619 06:50:26.209124 19640 ts_tablet_manager.cc:1397] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:50:26.211670 19637 raft_consensus.cc:348] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.212087 19637 raft_consensus.cc:374] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:26.212186 19637 raft_consensus.cc:729] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c958c4a3f5466a9b0f1130633f9573, State: Initialized, Role: FOLLOWER
I20260619 06:50:26.212770 19637 consensus_queue.cc:260] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.214000 19486 heartbeater.cc:503] Master 127.18.39.126:38763 was elected leader, sending a full tablet report...
I20260619 06:50:26.214581 19637 ts_tablet_manager.cc:1428] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Time spent starting tablet: real 0.012s user 0.005s sys 0.006s
I20260619 06:50:26.216924 19638 raft_consensus.cc:348] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.217429 19638 raft_consensus.cc:374] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:26.217582 19638 raft_consensus.cc:729] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d09a5ceb11d344f8b985e2a23c0e7733, State: Initialized, Role: FOLLOWER
I20260619 06:50:26.218278 19638 consensus_queue.cc:260] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.219015 19640 raft_consensus.cc:348] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.219414 19640 raft_consensus.cc:374] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:26.219513 19640 raft_consensus.cc:729] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 708df65d49814b3fa0ca8d230437d56d, State: Initialized, Role: FOLLOWER
I20260619 06:50:26.220093 19640 consensus_queue.cc:260] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.220494 19619 heartbeater.cc:503] Master 127.18.39.126:38763 was elected leader, sending a full tablet report...
I20260619 06:50:26.221319 19638 ts_tablet_manager.cc:1428] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260619 06:50:26.222790 19640 ts_tablet_manager.cc:1428] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260619 06:50:26.232687 19643 raft_consensus.cc:482] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:26.233003 19643 raft_consensus.cc:504] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.234701 19643 leader_election.cc:288] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d09a5ceb11d344f8b985e2a23c0e7733 (127.18.39.67:45591), 708df65d49814b3fa0ca8d230437d56d (127.18.39.65:34855)
I20260619 06:50:26.243407 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.235091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:56947 (local address 127.18.39.67:45591)
0619 06:50:26.235292 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.235298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.236055 (+ 757us) server_negotiation.cc:407] Connection header received
0619 06:50:26.236536 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.236540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.236591 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.236654 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:26.237215 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.238283 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.240967 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.242226 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.242390 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.242463 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.243080 (+ 617us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.243204 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:50:26.243496 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.235155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:44149 (local address 127.18.39.65:34855)
0619 06:50:26.235323 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.235330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.236096 (+ 766us) server_negotiation.cc:407] Connection header received
0619 06:50:26.236264 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.236268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.236324 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.236398 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:26.236965 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.238084 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.240973 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.242215 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.242399 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.242463 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.243112 (+ 649us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.243276 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:50:26.243549 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.234940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:45591 (local address 127.18.39.66:56947)
0619 06:50:26.235938 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.235960 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.236124 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.236703 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.236713 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.236743 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:26.237082 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.237089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.238435 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.238440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.240797 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.240812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.242511 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.242518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.242810 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.242924 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.243026 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.243159 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":100,"spinlock_wait_cycles":22528,"thread_start_us":603,"threads_started":1}
I20260619 06:50:26.244313 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.234937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:34855 (local address 127.18.39.66:44149)
0619 06:50:26.235945 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:26.235960 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:26.236125 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:26.236426 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:26.236430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:26.236449 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:26.236804 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.236819 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.238260 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.238264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.240797 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:26.240812 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.242511 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.242521 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:26.242810 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.242929 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 06:50:26.243746 (+ 817us) client_negotiation.cc:240] Negotiation successful
0619 06:50:26.243908 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":225,"spinlock_wait_cycles":29312,"thread_start_us":556,"threads_started":1}
I20260619 06:50:26.244673 19574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6129db7ebb4a6b8b855d4b2f274bf0" candidate_uuid: "97c958c4a3f5466a9b0f1130633f9573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" is_pre_election: true
I20260619 06:50:26.245240 19574 raft_consensus.cc:2454] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c958c4a3f5466a9b0f1130633f9573 in term 0.
I20260619 06:50:26.245249 19307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6129db7ebb4a6b8b855d4b2f274bf0" candidate_uuid: "97c958c4a3f5466a9b0f1130633f9573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708df65d49814b3fa0ca8d230437d56d" is_pre_election: true
I20260619 06:50:26.245810 19307 raft_consensus.cc:2454] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c958c4a3f5466a9b0f1130633f9573 in term 0.
I20260619 06:50:26.245949 19376 leader_election.cc:302] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c958c4a3f5466a9b0f1130633f9573, d09a5ceb11d344f8b985e2a23c0e7733; no voters:
I20260619 06:50:26.246378 19643 raft_consensus.cc:2781] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:26.246491 19643 raft_consensus.cc:482] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:26.246635 19643 raft_consensus.cc:3037] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:26.248773 19643 raft_consensus.cc:504] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.249419 19643 leader_election.cc:288] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [CANDIDATE]: Term 1 election: Requested vote from peers d09a5ceb11d344f8b985e2a23c0e7733 (127.18.39.67:45591), 708df65d49814b3fa0ca8d230437d56d (127.18.39.65:34855)
I20260619 06:50:26.249871 19574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6129db7ebb4a6b8b855d4b2f274bf0" candidate_uuid: "97c958c4a3f5466a9b0f1130633f9573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d09a5ceb11d344f8b985e2a23c0e7733"
I20260619 06:50:26.249922 19307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d6129db7ebb4a6b8b855d4b2f274bf0" candidate_uuid: "97c958c4a3f5466a9b0f1130633f9573" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "708df65d49814b3fa0ca8d230437d56d"
I20260619 06:50:26.250083 19574 raft_consensus.cc:3037] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:26.250100 19307 raft_consensus.cc:3037] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:26.253314 19307 raft_consensus.cc:2454] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c958c4a3f5466a9b0f1130633f9573 in term 1.
I20260619 06:50:26.253322 19574 raft_consensus.cc:2454] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c958c4a3f5466a9b0f1130633f9573 in term 1.
I20260619 06:50:26.253717 19377 leader_election.cc:302] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 708df65d49814b3fa0ca8d230437d56d, 97c958c4a3f5466a9b0f1130633f9573; no voters:
I20260619 06:50:26.254092 19643 raft_consensus.cc:2781] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:26.254483 19643 raft_consensus.cc:686] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [term 1 LEADER]: Becoming Leader. State: Replica: 97c958c4a3f5466a9b0f1130633f9573, State: Running, Role: LEADER
I20260619 06:50:26.255008 19643 consensus_queue.cc:237] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } }
I20260619 06:50:26.259407 19162 catalog_manager.cc:5495] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573 reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c958c4a3f5466a9b0f1130633f9573 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "97c958c4a3f5466a9b0f1130633f9573" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97c958c4a3f5466a9b0f1130633f9573" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 46845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d09a5ceb11d344f8b985e2a23c0e7733" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "708df65d49814b3fa0ca8d230437d56d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 34855 } health_report { overall_health: UNKNOWN } } }
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851824, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:26.282140 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.267571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.18.39.65:34855)
0619 06:50:26.267746 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.267751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.268312 (+ 561us) server_negotiation.cc:407] Connection header received
0619 06:50:26.268432 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.268436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.268498 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.268577 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.270166 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.270876 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.272066 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.272472 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.272559 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.276700 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.276720 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.276741 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.276796 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.279721 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.280357 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.280364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.280366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.280484 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.280786 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.280789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.280791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.281123 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.281400 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.281621 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.281773 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:26.327735 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.313461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55192 (local address 127.18.39.65:34855)
0619 06:50:26.313660 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.313668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.314036 (+ 368us) server_negotiation.cc:407] Connection header received
0619 06:50:26.314252 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.314256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.314323 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.314438 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.315973 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.316726 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.318222 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.318640 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.318776 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.322681 (+ 3905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.322686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.322689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.322723 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.325089 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.325788 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.325795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.325800 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.325927 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.326336 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.326350 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.326352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.326603 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.326857 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.327143 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.327375 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 06:50:26.334892 19620 tablet.cc:2369] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P d09a5ceb11d344f8b985e2a23c0e7733: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260619 06:50:26.358596 19487 tablet.cc:2369] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 97c958c4a3f5466a9b0f1130633f9573: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:26.368191 19353 tablet.cc:2369] T 8d6129db7ebb4a6b8b855d4b2f274bf0 P 708df65d49814b3fa0ca8d230437d56d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:26.380489 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.361189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.18.39.65:34855)
0619 06:50:26.361511 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.361517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.365576 (+ 4059us) server_negotiation.cc:407] Connection header received
0619 06:50:26.365879 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.365886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.365976 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.366048 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.368068 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.369150 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.370888 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.371318 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.371437 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.375404 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.375408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.375412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.375451 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.377941 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.378713 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.378717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.378720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.378838 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.379169 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.379172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.379174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.379461 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.379639 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.379930 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.380104 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
W20260619 06:50:26.382313 19267 tablet_service.cc:2908] Rejecting scan request for tablet 8d6129db7ebb4a6b8b855d4b2f274bf0: Uninitialized: clean time has not yet been initialized
I20260619 06:50:26.385659 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19224
I20260619 06:50:26.397933 19636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.386533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.18.39.65:34855)
0619 06:50:26.386684 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.386688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.387049 (+ 361us) server_negotiation.cc:407] Connection header received
0619 06:50:26.387198 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.387202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.387273 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.387363 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.388494 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.389148 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.390327 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.390730 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.390835 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.393005 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.393010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.393014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.393047 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.395643 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.396272 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.396275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.396277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.396403 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.396696 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.396699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.396701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.396969 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.397179 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.397520 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.397693 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:50:26.398702 19347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:26.488929 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19224
I20260619 06:50:26.506232 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19358
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:26.522519 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.507040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.18.39.66:46845)
0619 06:50:26.507219 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.507227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.507247 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:50:26.507467 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.507471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.507535 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.507618 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.509050 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.509819 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.511611 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.511994 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.512084 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.516691 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.516706 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.516722 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.516769 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.519618 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.520335 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.520343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.520367 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.520513 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.520902 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.520908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.520912 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.521260 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.521665 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.521930 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.522171 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:50:26.523166 19481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:26.611693 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19358
I20260619 06:50:26.628386 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19491
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:26.643184 19634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:26.629074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.18.39.67:45591)
0619 06:50:26.629232 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:26.629236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:26.629256 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:50:26.629444 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:26.629448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:26.629522 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:26.629607 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:26.630816 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.631540 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.632840 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:26.633245 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:26.633433 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:26.637710 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:26.637725 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:26.637742 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:26.637781 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:26.640247 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.640986 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.640993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.640996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.641132 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:26.641567 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:26.641573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:26.641576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:26.641950 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:26.642246 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:26.642572 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 06:50:26.642796 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:50:26.643841 19614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:26.732476 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19491
I20260619 06:50:26.748384 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19131
I20260619 06:50:26.749557 19193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:26.858713 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19131
2026-06-19T06:50:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4046 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19667](info): set up 2 sockets
Jun 19 06:50:26 dist-test-slave-1h5s krb5kdc[19667](info): commencing operation
krb5kdc: starting...
W20260619 06:50:27.937548 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:28 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:28.118958 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40645
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:28.449829 19680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:28.450148 19680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:28.450325 19680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:28.457958 19680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:28.458088 19680 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:50:28.458170 19680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:28.458268 19680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:28.458349 19680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:28.458457 19680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:28.468858 19680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40645
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:28.470273 19680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:28.471606 19680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:28.483274 19685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:28.483382 19688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:28.483408 19686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:28.484190 19680 server_base.cc:1034] running on GCE node
I20260619 06:50:28.485020 19680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:28.486438 19680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:28.487660 19680 hybrid_clock.cc:648] HybridClock initialized: now 1781851828487593 us; error 47 us; skew 500 ppm
I20260619 06:50:28.487999 19680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:28 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:28.491020 19680 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:28.492461 19680 webserver.cc:460] Webserver started at http://127.18.39.126:39241/ using document root <none> and password file <none>
I20260619 06:50:28.493175 19680 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:28.493309 19680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:28.493674 19680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:28.496816 19680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "020f7cf4e53046aab455516afe25dca4"
format_stamp: "Formatted at 2026-06-19 06:50:28 on dist-test-slave-1h5s"
server_key: "dca3f775ec6a39fa0689de4f6db96be5"
server_key_iv: "a22cf4dc4e9ef576098c35d34715c838"
server_key_version: "encryptionkey@0"
I20260619 06:50:28.497722 19680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "020f7cf4e53046aab455516afe25dca4"
format_stamp: "Formatted at 2026-06-19 06:50:28 on dist-test-slave-1h5s"
server_key: "dca3f775ec6a39fa0689de4f6db96be5"
server_key_iv: "a22cf4dc4e9ef576098c35d34715c838"
server_key_version: "encryptionkey@0"
I20260619 06:50:28.503237 19680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:28.506929 19695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:28.508314 19680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:50:28.508551 19680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "020f7cf4e53046aab455516afe25dca4"
format_stamp: "Formatted at 2026-06-19 06:50:28 on dist-test-slave-1h5s"
server_key: "dca3f775ec6a39fa0689de4f6db96be5"
server_key_iv: "a22cf4dc4e9ef576098c35d34715c838"
server_key_version: "encryptionkey@0"
I20260619 06:50:28.508845 19680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:28.547669 19680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:28.551890 19680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:28.552194 19680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:28.574555 19680 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:40645
I20260619 06:50:28.574597 19756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:40645 every 8 connection(s)
I20260619 06:50:28.575976 19680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:28.576602 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19680
I20260619 06:50:28.576939 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:28.577790 18589 external_mini_cluster.cc:1428] Setting key f689dd5fc64013d02ca3f465479341cf
I20260619 06:50:28.580363 19757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:50:28 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851828, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:28.594039 19757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Bootstrap starting.
I20260619 06:50:28.597234 19757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:28.598277 19757 log.cc:824] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:28.601800 19757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: No bootstrap required, opened a new log
I20260619 06:50:28.601856 19760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:28.579841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.18.39.126:40645)
0619 06:50:28.580654 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:28.580666 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:28.580706 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:50:28.582669 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:28.582692 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:28.583115 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:28.583473 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:28.585198 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:28.586651 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:28.588357 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:28.589018 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:28.589142 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:28.594081 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:28.594089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:28.594105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:28.594148 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:28.597171 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:28.597941 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:28.597949 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:28.597955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:28.598125 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:28.598536 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:28.598542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:28.598545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:28.599255 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:28.599496 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:28.600032 (+ 536us) server_negotiation.cc:299] Negotiation successful
0619 06:50:28.600492 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":357,"threads_started":1}
I20260619 06:50:28.609969 19757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } }
I20260619 06:50:28.610318 19757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:28.610396 19757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020f7cf4e53046aab455516afe25dca4, State: Initialized, Role: FOLLOWER
I20260619 06:50:28.610888 19757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } }
I20260619 06:50:28.611073 19757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:28.611182 19757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:28.611316 19757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:28.613961 19757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } }
I20260619 06:50:28.614535 19757 leader_election.cc:302] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 020f7cf4e53046aab455516afe25dca4; no voters:
I20260619 06:50:28.615089 19757 leader_election.cc:288] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:28.615281 19762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:28.615981 19762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [term 1 LEADER]: Becoming Leader. State: Replica: 020f7cf4e53046aab455516afe25dca4, State: Running, Role: LEADER
I20260619 06:50:28.616544 19762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } }
I20260619 06:50:28.617228 19757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:28.619719 19763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "020f7cf4e53046aab455516afe25dca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } } }
I20260619 06:50:28.619699 19764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 020f7cf4e53046aab455516afe25dca4. Latest consensus state: current_term: 1 leader_uuid: "020f7cf4e53046aab455516afe25dca4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "020f7cf4e53046aab455516afe25dca4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40645 } } }
I20260619 06:50:28.620177 19764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:28.620173 19763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:28.622678 19769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:28.629719 19769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:28.639323 19769 catalog_manager.cc:1347] Generated new cluster ID: c11ed537e076401788122252326cb042
I20260619 06:50:28.639457 19769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:28.647459 19769 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:28.648720 19769 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:28.654434 19769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Generated new TSK 0
I20260619 06:50:28.655243 19769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:28.697301 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40645
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:29.028952 19783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:29.029302 19783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:29.029501 19783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:29.036931 19783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:29.037098 19783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:29.037256 19783 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:29.037422 19783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:29.047742 19783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:29.049283 19783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:29.050871 19783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:29.064200 19788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:29.064285 19789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:29.064237 19791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:29.066839 19783 server_base.cc:1034] running on GCE node
I20260619 06:50:29.067454 19783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:29.068389 19783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:29.069636 19783 hybrid_clock.cc:648] HybridClock initialized: now 1781851829069588 us; error 47 us; skew 500 ppm
I20260619 06:50:29.069952 19783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:29 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851829, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:29.076498 19783 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:29.078189 19783 webserver.cc:460] Webserver started at http://127.18.39.65:34021/ using document root <none> and password file <none>
I20260619 06:50:29.078960 19783 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:29.079093 19783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:29.079433 19783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:29.082509 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "4f65e692de82440e886d62610dd6ba0b"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "73af4ee554428cc8e9f7f107ddd87b44"
server_key_iv: "17539f5c1b2b28c73d44399faf109d48"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.083328 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "4f65e692de82440e886d62610dd6ba0b"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "73af4ee554428cc8e9f7f107ddd87b44"
server_key_iv: "17539f5c1b2b28c73d44399faf109d48"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.088464 19783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:29.091931 19798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:29.093242 19783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:29.093451 19783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "4f65e692de82440e886d62610dd6ba0b"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "73af4ee554428cc8e9f7f107ddd87b44"
server_key_iv: "17539f5c1b2b28c73d44399faf109d48"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.093673 19783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:29.133736 19783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:29.138043 19783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:29.138350 19783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:29.141059 19783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:29.141196 19783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:29.141413 19783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:29.141496 19783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Jun 19 06:50:29 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851829, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:29.166146 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.141849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:51831 (local address 127.18.39.126:40645)
0619 06:50:29.142457 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:29.142463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:29.143623 (+ 1160us) server_negotiation.cc:407] Connection header received
0619 06:50:29.146725 (+ 3102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:29.146732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:29.146826 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:29.146934 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:29.149445 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.150445 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.152911 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.153707 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.153896 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.159332 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:29.159341 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:29.159348 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:29.159392 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:29.162166 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.163349 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.163353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.163355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.163473 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.164047 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.164050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.164052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.164329 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:29.164438 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:29.165653 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0619 06:50:29.165888 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":346,"threads_started":1}
I20260619 06:50:29.168511 19808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.142566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.65:51831)
0619 06:50:29.143425 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:29.143478 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:29.146348 (+ 2870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:29.147260 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:29.147276 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:29.148198 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:29.149181 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.149212 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.150597 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.150602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.152714 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.152737 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.153998 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.154011 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.154406 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.155302 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:29.155334 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:29.158973 (+ 3639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:29.162323 (+ 3350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.162337 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.162361 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.163177 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.163587 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.163595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.163601 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.163926 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.164459 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:29.164471 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:29.165113 (+ 642us) client_negotiation.cc:769] Sending connection context
0619 06:50:29.165541 (+ 428us) client_negotiation.cc:240] Negotiation successful
0619 06:50:29.166312 (+ 771us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":334,"threads_started":1}
I20260619 06:50:29.191460 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.174734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.65:46403)
0619 06:50:29.175525 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:29.175551 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:29.175742 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:29.176084 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:29.176092 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:29.176869 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:29.177413 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.177438 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.178714 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.178723 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.181016 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.181036 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.182357 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.182367 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.182617 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.183426 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:29.183457 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:29.184622 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:29.188834 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.188844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.188851 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.189249 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.189774 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.189778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.189780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.189894 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.190597 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:29.190600 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:29.190754 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 06:50:29.190868 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 06:50:29.191131 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":368,"threads_started":1}
I20260619 06:50:29.191840 19807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.174728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46403 (local address 127.18.39.126:40645)
0619 06:50:29.174960 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:29.174967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:29.175659 (+ 692us) server_negotiation.cc:407] Connection header received
0619 06:50:29.175868 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:29.175871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:29.175935 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:29.176016 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:29.177610 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.178559 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.181242 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.182082 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.182236 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.184901 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:29.184908 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:29.184913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:29.184950 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:29.188644 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.189432 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.189441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.189447 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.189652 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.190004 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.190011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.190017 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.190470 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:29.190650 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:29.191028 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 06:50:29.191342 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:50:29.198541 19783 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:35177
I20260619 06:50:29.198560 19918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:35177 every 8 connection(s)
I20260619 06:50:29.199735 19783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:29.201689 19919 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40645
I20260619 06:50:29.201922 19919 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:29.202543 19919 heartbeater.cc:511] Master 127.18.39.126:40645 requested a full tablet report, sending...
I20260619 06:50:29.204425 19712 ts_manager.cc:194] Registered new tserver with Master: 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177)
I20260619 06:50:29.205937 19712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:51831
I20260619 06:50:29.207428 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19783
I20260619 06:50:29.207644 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:29.208274 18589 external_mini_cluster.cc:1428] Setting key 598564cf7e68a6e2c3dddb2df7f2516e
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:29.245707 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40645
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:29.574484 19923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:29.574807 19923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:29.574975 19923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:29.582417 19923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:29.582574 19923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:29.582726 19923 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:29.582859 19923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:29.592988 19923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:29.594380 19923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:29.595685 19923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:29.609937 19928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:29.609858 19931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:29.609911 19929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:29.615748 19923 server_base.cc:1034] running on GCE node
I20260619 06:50:29.616483 19923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:29.617617 19923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:29.618863 19923 hybrid_clock.cc:648] HybridClock initialized: now 1781851829618807 us; error 44 us; skew 500 ppm
I20260619 06:50:29.619132 19923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:29 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851829, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:29.622418 19923 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:29.624085 19923 webserver.cc:460] Webserver started at http://127.18.39.66:36705/ using document root <none> and password file <none>
I20260619 06:50:29.624857 19923 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:29.624997 19923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:29.625401 19923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:29.628345 19923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "a866bcf10bad4c86852bcc8f8d6b9176"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "4f8239ec0acf681ba474b31b20011061"
server_key_iv: "074228f94eb620b00366f856997da262"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.629220 19923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "a866bcf10bad4c86852bcc8f8d6b9176"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "4f8239ec0acf681ba474b31b20011061"
server_key_iv: "074228f94eb620b00366f856997da262"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.634673 19923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:50:29.638286 19938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:29.639793 19923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:29.639993 19923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "a866bcf10bad4c86852bcc8f8d6b9176"
format_stamp: "Formatted at 2026-06-19 06:50:29 on dist-test-slave-1h5s"
server_key: "4f8239ec0acf681ba474b31b20011061"
server_key_iv: "074228f94eb620b00366f856997da262"
server_key_version: "encryptionkey@0"
I20260619 06:50:29.640267 19923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:29.684634 19923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:29.688674 19923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:29.688989 19923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:29.691704 19923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:29.691839 19923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:29.691987 19923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:29.692041 19923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:50:29 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851829, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:29.719247 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.692672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:45897 (local address 127.18.39.126:40645)
0619 06:50:29.693164 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:29.693171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:29.694581 (+ 1410us) server_negotiation.cc:407] Connection header received
0619 06:50:29.697811 (+ 3230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:29.697816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:29.697923 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:29.698090 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:29.701528 (+ 3438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.702952 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.705784 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.706214 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.706356 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.711525 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:29.711531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:29.711534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:29.711568 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:29.714658 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.715830 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.715840 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.715846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.716018 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.716684 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.716693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.716699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.717125 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:29.717322 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:29.718464 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0619 06:50:29.718740 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":266,"threads_started":1}
I20260619 06:50:29.720796 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.693468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.66:45897)
0619 06:50:29.694383 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:29.694439 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:29.697404 (+ 2965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:29.698627 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:29.698644 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:29.699921 (+ 1277us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:29.701241 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.701277 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.703154 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.703166 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.705598 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.705620 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.706454 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.706463 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.706824 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.707830 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:29.707860 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:29.711175 (+ 3315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:29.714801 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.714812 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.714840 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.715616 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.716171 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.716181 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.716186 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.716521 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.717314 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:29.717329 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:29.717902 (+ 573us) client_negotiation.cc:769] Sending connection context
0619 06:50:29.718275 (+ 373us) client_negotiation.cc:240] Negotiation successful
0619 06:50:29.718958 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":395,"threads_started":1}
I20260619 06:50:29.737937 19923 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:39437
I20260619 06:50:29.737938 20058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:39437 every 8 connection(s)
I20260619 06:50:29.739619 19923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:29.742130 20059 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40645
I20260619 06:50:29.742373 20059 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:29.743237 20059 heartbeater.cc:511] Master 127.18.39.126:40645 requested a full tablet report, sending...
I20260619 06:50:29.744637 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 19923
I20260619 06:50:29.744848 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:29.744817 19712 ts_manager.cc:194] Registered new tserver with Master: a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66:39437)
I20260619 06:50:29.745069 20037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.727887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.66:47065)
0619 06:50:29.728630 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:29.728654 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:29.728839 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:29.729521 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:29.729528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:29.730434 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:29.730955 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.730975 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.732813 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.732816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.734494 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:29.734513 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.735876 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.735882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:29.736048 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.736745 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:29.736770 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:29.738161 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:29.742082 (+ 3921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.742087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.742092 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.742637 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.743047 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:29.743050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:29.743052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:29.743243 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:29.743953 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:29.743958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:29.744115 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 06:50:29.744324 (+ 209us) client_negotiation.cc:240] Negotiation successful
0619 06:50:29.744619 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":445,"threads_started":1}
I20260619 06:50:29.745443 18589 external_mini_cluster.cc:1428] Setting key 65a813c620e542318e5e99310a2b3a4b
I20260619 06:50:29.745877 19712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:45897
I20260619 06:50:29.745932 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:29.727817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47065 (local address 127.18.39.126:40645)
0619 06:50:29.728047 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:29.728055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:29.728754 (+ 699us) server_negotiation.cc:407] Connection header received
0619 06:50:29.729021 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:29.729029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:29.729123 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:29.729256 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:29.731152 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.732645 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.734670 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:29.735496 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:29.735624 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:29.738406 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:29.738412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:29.738417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:29.738453 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:29.741939 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.742775 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.742782 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.742787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.742939 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:29.743401 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:29.743406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:29.743409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:29.743830 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:29.743996 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:29.745520 (+ 1524us) server_negotiation.cc:299] Negotiation successful
0619 06:50:29.745725 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:29.784276 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40645
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:30.109071 20063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:30.109463 20063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:30.109623 20063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:30.116995 20063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:30.117110 20063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:30.117296 20063 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:30.117460 20063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:30.127645 20063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:30.129068 20063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:30.130594 20063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:30.143882 20071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:30.143929 20069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:30.144374 20068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:30.145298 20063 server_base.cc:1034] running on GCE node
I20260619 06:50:30.146025 20063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:30.146999 20063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:30.148296 20063 hybrid_clock.cc:648] HybridClock initialized: now 1781851830148224 us; error 45 us; skew 500 ppm
I20260619 06:50:30.148619 20063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:30 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:30.151914 20063 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:30.153558 20063 webserver.cc:460] Webserver started at http://127.18.39.67:38361/ using document root <none> and password file <none>
I20260619 06:50:30.154269 20063 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:30.154402 20063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:30.154731 20063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:30.157804 20063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "81a2a59deb15495ea9f0cf9fe7705929"
format_stamp: "Formatted at 2026-06-19 06:50:30 on dist-test-slave-1h5s"
server_key: "83125638e2f39e28b30f192327634179"
server_key_iv: "3ca2cafeaa1f3bbd841e7df0dc3ee939"
server_key_version: "encryptionkey@0"
I20260619 06:50:30.158679 20063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "81a2a59deb15495ea9f0cf9fe7705929"
format_stamp: "Formatted at 2026-06-19 06:50:30 on dist-test-slave-1h5s"
server_key: "83125638e2f39e28b30f192327634179"
server_key_iv: "3ca2cafeaa1f3bbd841e7df0dc3ee939"
server_key_version: "encryptionkey@0"
I20260619 06:50:30.164031 20063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:50:30.167655 20078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:30.169091 20063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:50:30.169319 20063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "81a2a59deb15495ea9f0cf9fe7705929"
format_stamp: "Formatted at 2026-06-19 06:50:30 on dist-test-slave-1h5s"
server_key: "83125638e2f39e28b30f192327634179"
server_key_iv: "3ca2cafeaa1f3bbd841e7df0dc3ee939"
server_key_version: "encryptionkey@0"
I20260619 06:50:30.169620 20063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:30.187108 20063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:30.191422 20063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:30.191711 20063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:30.194500 20063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:30.194613 20063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:30.194813 20063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:30.194905 20063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:30.208649 19919 heartbeater.cc:503] Master 127.18.39.126:40645 was elected leader, sending a full tablet report...
Jun 19 06:50:30 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:30.220882 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.195549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.67:42973)
0619 06:50:30.196331 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.196369 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.198472 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.199541 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.199553 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.200442 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:30.201505 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.201538 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.203259 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.203266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.205462 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.205483 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.206391 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.206400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.206638 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.207404 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:30.207447 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:30.211036 (+ 3589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:30.215574 (+ 4538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.215587 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.215610 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.216111 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.216640 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.216643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.216646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.216836 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.217612 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:30.217622 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:30.218124 (+ 502us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.218491 (+ 367us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.219139 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":330,"threads_started":1}
I20260619 06:50:30.221935 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.194833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:42973 (local address 127.18.39.126:40645)
0619 06:50:30.195016 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.195021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.196489 (+ 1468us) server_negotiation.cc:407] Connection header received
0619 06:50:30.198873 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.198879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.198962 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.199022 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:30.201749 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.203022 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.205675 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.206141 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.206289 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.211361 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:30.211369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:30.211377 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:30.211422 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:30.215392 (+ 3970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.216298 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.216307 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.216313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.216514 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.216957 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.216965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.216971 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.217453 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:30.217612 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.221518 (+ 3906us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.221691 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:50:30.244756 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.229288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:35175 (local address 127.18.39.126:40645)
0619 06:50:30.229556 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.229565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.230158 (+ 593us) server_negotiation.cc:407] Connection header received
0619 06:50:30.230366 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.230374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.230497 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.230604 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:30.231615 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.233134 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.235254 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.235955 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.236099 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.238591 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:30.238601 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:30.238608 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:30.238658 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:30.241993 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.242696 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.242702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.242710 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.242848 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.243229 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.243234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.243242 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.243513 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:30.243677 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.244135 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.244377 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 06:50:30.244921 20167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.229395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.18.39.67:35175)
0619 06:50:30.230050 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.230073 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.230187 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.230621 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.230624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.231095 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:30.231472 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.231482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.233305 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.233309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.235000 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.235015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.236140 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.236145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.236307 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.237061 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:30.237093 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:30.238364 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:30.242145 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.242150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.242154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.242558 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.242962 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.242969 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.242973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.243107 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.243693 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:30.243699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:30.243876 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.244056 (+ 180us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.244397 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":341,"threads_started":1}
I20260619 06:50:30.248611 20063 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44709
I20260619 06:50:30.248642 20197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44709 every 8 connection(s)
I20260619 06:50:30.249967 20063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:30.251935 20198 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40645
I20260619 06:50:30.251981 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 20063
I20260619 06:50:30.252169 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:30.252170 20198 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:30.252642 18589 external_mini_cluster.cc:1428] Setting key a9387c12c8d9b402992533090d496b53
I20260619 06:50:30.252859 20198 heartbeater.cc:511] Master 127.18.39.126:40645 requested a full tablet report, sending...
I20260619 06:50:30.254312 19711 ts_manager.cc:194] Registered new tserver with Master: 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.255277 19711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:42973
I20260619 06:50:30.255738 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:50:30 dist-test-slave-1h5s krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:50:30 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:30.300093 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.286764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56822 (local address 127.18.39.126:40645)
0619 06:50:30.287010 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.287016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.287210 (+ 194us) server_negotiation.cc:407] Connection header received
0619 06:50:30.287360 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.287364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.287425 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.287509 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:30.288864 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.289654 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.290912 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.291307 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.291396 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.295309 (+ 3913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:30.295314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:30.295317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:30.295351 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:30.297876 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.298507 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.298511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.298514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.298629 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.298955 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.298958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.298960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.299200 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:30.299376 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.299652 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.299819 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 06:50:30.304373 19711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:30.311708 19711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:30.337462 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.324498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45168 (local address 127.18.39.66:39437)
0619 06:50:30.325380 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.325391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.326274 (+ 883us) server_negotiation.cc:407] Connection header received
0619 06:50:30.326396 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.326408 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.326718 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.327189 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.327857 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.329866 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.333638 (+ 3772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.335646 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.335832 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.336083 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.336778 (+ 695us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.337032 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":577,"threads_started":1}
I20260619 06:50:30.337271 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.324357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39437 (local address 127.0.0.1:45168)
0619 06:50:30.325804 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.325847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.326047 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.326996 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.327008 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.327050 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.327594 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.327611 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.330023 (+ 2412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.330031 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.333431 (+ 3400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.333453 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.336221 (+ 2768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.336227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.336465 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.336531 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.336671 (+ 140us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.336887 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1191,"thread_start_us":12175,"threads_started":1}
I20260619 06:50:30.345618 19989 tablet_service.cc:1467] Processing CreateTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c (DEFAULT_TABLE table=test-table [id=b085c4dc1b7147fe8ea6832223290348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.347198 19989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ede1b5f0e2c43889340d14b6610e25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.349162 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.341492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:35177 (local address 127.0.0.1:53956)
0619 06:50:30.342225 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.342245 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.342329 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.343396 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.343401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.343424 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.343613 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.343623 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.345301 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.345306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.346770 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.346786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.348290 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.348295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.348468 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.348535 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.348662 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.348862 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":27}
I20260619 06:50:30.349529 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.325057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44709 (local address 127.0.0.1:44616)
0619 06:50:30.336303 (+ 11246us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.336323 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.336445 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.341772 (+ 5327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.341777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.341806 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.342155 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.342166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.344211 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.344217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.346384 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.346404 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.348693 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.348698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.348875 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.348902 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.348989 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.349107 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11088,"mutex_wait_us":4,"thread_start_us":442,"threads_started":1}
I20260619 06:50:30.349915 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.325755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.18.39.67:44709)
0619 06:50:30.341037 (+ 15282us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.341044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.341073 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:50:30.341153 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.341159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.341453 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.341814 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.342399 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.344060 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.346591 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.348486 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.348623 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.348842 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.349061 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.349301 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15127,"thread_start_us":14946,"threads_started":1}
I20260619 06:50:30.350102 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.337173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.18.39.65:35177)
0619 06:50:30.342821 (+ 5648us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.342830 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.342873 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:50:30.342971 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.342979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.343197 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.343850 (+ 653us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.344029 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.345151 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.346944 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.348120 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.348276 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.348482 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.349650 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.349827 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5472,"thread_start_us":5344,"threads_started":1}
I20260619 06:50:30.353745 20128 tablet_service.cc:1467] Processing CreateTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c (DEFAULT_TABLE table=test-table [id=b085c4dc1b7147fe8ea6832223290348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.353745 19849 tablet_service.cc:1467] Processing CreateTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c (DEFAULT_TABLE table=test-table [id=b085c4dc1b7147fe8ea6832223290348]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.356278 20128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ede1b5f0e2c43889340d14b6610e25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.363703 20216 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Bootstrap starting.
I20260619 06:50:30.364025 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ede1b5f0e2c43889340d14b6610e25c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.367599 20216 tablet_bootstrap.cc:654] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.370832 20216 log.cc:824] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:30.373777 20218 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Bootstrap starting.
I20260619 06:50:30.378566 20218 tablet_bootstrap.cc:654] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.380229 20218 log.cc:824] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:30.381055 20216 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: No bootstrap required, opened a new log
I20260619 06:50:30.381453 20216 ts_tablet_manager.cc:1397] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260619 06:50:30.385607 20219 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Bootstrap starting.
I20260619 06:50:30.389288 20218 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: No bootstrap required, opened a new log
I20260619 06:50:30.389695 20218 ts_tablet_manager.cc:1397] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260619 06:50:30.390889 20219 tablet_bootstrap.cc:654] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.392282 20219 log.cc:824] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:30.393031 20216 raft_consensus.cc:348] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.393486 20216 raft_consensus.cc:374] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.393599 20216 raft_consensus.cc:729] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a866bcf10bad4c86852bcc8f8d6b9176, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.394333 20216 consensus_queue.cc:260] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.395521 20059 heartbeater.cc:503] Master 127.18.39.126:40645 was elected leader, sending a full tablet report...
I20260619 06:50:30.396008 20219 tablet_bootstrap.cc:492] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: No bootstrap required, opened a new log
I20260619 06:50:30.396178 20216 ts_tablet_manager.cc:1428] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260619 06:50:30.396306 20219 ts_tablet_manager.cc:1397] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260619 06:50:30.401046 20218 raft_consensus.cc:348] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.401649 20218 raft_consensus.cc:374] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.401795 20218 raft_consensus.cc:729] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a2a59deb15495ea9f0cf9fe7705929, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.402634 20218 consensus_queue.cc:260] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.404490 20198 heartbeater.cc:503] Master 127.18.39.126:40645 was elected leader, sending a full tablet report...
I20260619 06:50:30.405710 20218 ts_tablet_manager.cc:1428] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260619 06:50:30.407677 20219 raft_consensus.cc:348] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.408146 20219 raft_consensus.cc:374] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.408303 20219 raft_consensus.cc:729] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f65e692de82440e886d62610dd6ba0b, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.408931 20219 consensus_queue.cc:260] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.410449 20219 ts_tablet_manager.cc:1428] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260619 06:50:30.452729 19920 tablet.cc:2369] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:30.462823 20222 raft_consensus.cc:482] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:30.463192 20222 raft_consensus.cc:504] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.465093 20222 leader_election.cc:288] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.472208 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.465475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:56229 (local address 127.18.39.65:35177)
0619 06:50:30.465694 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.465702 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.466225 (+ 523us) server_negotiation.cc:407] Connection header received
0619 06:50:30.466440 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.466445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.466549 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.466647 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.467341 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.468301 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.470247 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.471172 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.471375 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.471454 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.471816 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.471945 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:50:30.472246 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.465805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:34601 (local address 127.18.39.67:44709)
0619 06:50:30.465986 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.465991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.466594 (+ 603us) server_negotiation.cc:407] Connection header received
0619 06:50:30.466774 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.466782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.466864 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.466957 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.467799 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.468538 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.470528 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.471282 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.471482 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.471553 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.471869 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.471995 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:50:30.472400 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.465562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44709 (local address 127.18.39.66:34601)
0619 06:50:30.466481 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.466498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.466632 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.466979 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.466982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.467001 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.467601 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.467614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.468700 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.468704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.470340 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.470353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.471573 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.471583 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.471712 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.471734 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.471844 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.471985 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":4,"thread_start_us":579,"threads_started":1}
I20260619 06:50:30.472524 20225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.465394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:35177 (local address 127.18.39.66:56229)
0619 06:50:30.466076 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.466102 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.466287 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.466677 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.466680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.466723 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.467099 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.467115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.468430 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.468435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.470081 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.470093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.471405 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.471411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.471651 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.471676 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.471790 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.471941 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":2,"spinlock_wait_cycles":7936,"thread_start_us":471,"threads_started":1}
I20260619 06:50:30.473577 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ede1b5f0e2c43889340d14b6610e25c" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929" is_pre_election: true
I20260619 06:50:30.473538 19874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ede1b5f0e2c43889340d14b6610e25c" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f65e692de82440e886d62610dd6ba0b" is_pre_election: true
I20260619 06:50:30.474087 19874 raft_consensus.cc:2454] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 0.
I20260619 06:50:30.474088 20148 raft_consensus.cc:2454] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 0.
I20260619 06:50:30.474822 19941 leader_election.cc:302] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f65e692de82440e886d62610dd6ba0b, a866bcf10bad4c86852bcc8f8d6b9176; no voters:
I20260619 06:50:30.475266 20222 raft_consensus.cc:2781] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:30.475416 20222 raft_consensus.cc:482] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:30.475519 20222 raft_consensus.cc:3037] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.477794 20222 raft_consensus.cc:504] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.478487 20222 leader_election.cc:288] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 election: Requested vote from peers 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.478914 19874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ede1b5f0e2c43889340d14b6610e25c" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f65e692de82440e886d62610dd6ba0b"
I20260619 06:50:30.479054 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ede1b5f0e2c43889340d14b6610e25c" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929"
I20260619 06:50:30.479130 19874 raft_consensus.cc:3037] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.479282 20148 raft_consensus.cc:3037] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.481616 20148 raft_consensus.cc:2454] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 1.
I20260619 06:50:30.481627 19874 raft_consensus.cc:2454] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 1.
I20260619 06:50:30.482098 19942 leader_election.cc:302] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a2a59deb15495ea9f0cf9fe7705929, a866bcf10bad4c86852bcc8f8d6b9176; no voters:
I20260619 06:50:30.482419 20222 raft_consensus.cc:2781] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:30.482817 20222 raft_consensus.cc:686] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 LEADER]: Becoming Leader. State: Replica: a866bcf10bad4c86852bcc8f8d6b9176, State: Running, Role: LEADER
I20260619 06:50:30.483385 20222 consensus_queue.cc:237] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.488133 19711 catalog_manager.cc:5495] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 reported cstate change: term changed from 0 to 1, leader changed from <none> to a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } health_report { overall_health: UNKNOWN } } }
W20260619 06:50:30.491897 20060 tablet.cc:2369] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:30.501575 20199 tablet.cc:2369] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:30.512013 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.506992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.18.39.66:39437)
0619 06:50:30.507178 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.507182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.507200 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:50:30.507348 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.507352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.507434 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.507546 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:30.507847 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.508535 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.509901 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.510300 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.510390 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.510879 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.511431 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.511533 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.511644 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.511707 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:50:30.521138 19874 raft_consensus.cc:1261] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Refusing update from remote peer a866bcf10bad4c86852bcc8f8d6b9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.522065 20148 raft_consensus.cc:1261] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Refusing update from remote peer a866bcf10bad4c86852bcc8f8d6b9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.522166 20227 consensus_queue.cc:1035] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:30.522845 20222 consensus_queue.cc:1035] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:30.552731 19712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:30.553016 19712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:30.556643 19712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Sending DeleteTablet for 3 replicas of tablet 5ede1b5f0e2c43889340d14b6610e25c
I20260619 06:50:30.557906 19989 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:30 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:45168
I20260619 06:50:30.558070 20128 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:30 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:44616
I20260619 06:50:30.558386 19849 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ede1b5f0e2c43889340d14b6610e25c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:30 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53956
I20260619 06:50:30.560151 19712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:30.561043 19712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:30.563922 20240 tablet_replica.cc:330] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: stopping tablet replica
I20260619 06:50:30.564445 20240 raft_consensus.cc:2229] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:30.565176 20240 raft_consensus.cc:2258] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:30.567423 20240 ts_tablet_manager.cc:1905] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:30.574411 19848 tablet_service.cc:1467] Processing CreateTablet for tablet 98332db5eaa7403f845abe5de3048e90 (DEFAULT_TABLE table=test-table [id=fc00d218e5194460a8efb5cef362923c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.575011 19848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98332db5eaa7403f845abe5de3048e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.575528 20242 tablet_replica.cc:330] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: stopping tablet replica
I20260619 06:50:30.576050 20242 raft_consensus.cc:2229] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:30.576617 20242 raft_consensus.cc:2258] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:30.577981 20240 ts_tablet_manager.cc:1918] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:30.578223 20240 log.cc:1196] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/5ede1b5f0e2c43889340d14b6610e25c
I20260619 06:50:30.578853 20240 ts_tablet_manager.cc:1939] T 5ede1b5f0e2c43889340d14b6610e25c P a866bcf10bad4c86852bcc8f8d6b9176: Deleting consensus metadata
I20260619 06:50:30.581419 20127 tablet_service.cc:1467] Processing CreateTablet for tablet 98332db5eaa7403f845abe5de3048e90 (DEFAULT_TABLE table=test-table [id=fc00d218e5194460a8efb5cef362923c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.582007 20127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98332db5eaa7403f845abe5de3048e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.583117 20242 ts_tablet_manager.cc:1905] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:30.585661 19698 catalog_manager.cc:4841] TS a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66:39437): tablet 5ede1b5f0e2c43889340d14b6610e25c (table test-table [id=b085c4dc1b7147fe8ea6832223290348]) successfully deleted
I20260619 06:50:30.586496 19989 tablet_service.cc:1467] Processing CreateTablet for tablet 98332db5eaa7403f845abe5de3048e90 (DEFAULT_TABLE table=test-table [id=fc00d218e5194460a8efb5cef362923c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:30.586985 19989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98332db5eaa7403f845abe5de3048e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.589586 20218 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Bootstrap starting.
I20260619 06:50:30.591215 20218 tablet_bootstrap.cc:654] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.592232 20242 ts_tablet_manager.cc:1918] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:30.592454 20242 log.cc:1196] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/5ede1b5f0e2c43889340d14b6610e25c
I20260619 06:50:30.594558 20242 ts_tablet_manager.cc:1939] T 5ede1b5f0e2c43889340d14b6610e25c P 4f65e692de82440e886d62610dd6ba0b: Deleting consensus metadata
I20260619 06:50:30.595191 20219 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Bootstrap starting.
I20260619 06:50:30.596591 19698 catalog_manager.cc:4841] TS 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177): tablet 5ede1b5f0e2c43889340d14b6610e25c (table test-table [id=b085c4dc1b7147fe8ea6832223290348]) successfully deleted
I20260619 06:50:30.596782 20219 tablet_bootstrap.cc:654] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.597314 20216 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Bootstrap starting.
I20260619 06:50:30.598910 20216 tablet_bootstrap.cc:654] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.602005 20241 tablet_replica.cc:330] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: stopping tablet replica
I20260619 06:50:30.602771 20241 raft_consensus.cc:2229] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:30.603417 20241 raft_consensus.cc:2258] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:30.605934 20218 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: No bootstrap required, opened a new log
I20260619 06:50:30.606149 20218 ts_tablet_manager.cc:1397] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260619 06:50:30.607466 20241 ts_tablet_manager.cc:1905] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:30.608613 20219 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: No bootstrap required, opened a new log
I20260619 06:50:30.608829 20219 ts_tablet_manager.cc:1397] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260619 06:50:30.609710 20219 raft_consensus.cc:348] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.609998 20219 raft_consensus.cc:374] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.610149 20219 raft_consensus.cc:729] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f65e692de82440e886d62610dd6ba0b, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.610500 20216 tablet_bootstrap.cc:492] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: No bootstrap required, opened a new log
I20260619 06:50:30.610678 20216 ts_tablet_manager.cc:1397] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.006s
I20260619 06:50:30.610494 20219 consensus_queue.cc:260] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.608608 20218 raft_consensus.cc:348] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.613029 20218 raft_consensus.cc:374] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.613145 20218 raft_consensus.cc:729] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a2a59deb15495ea9f0cf9fe7705929, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.613510 20218 consensus_queue.cc:260] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.613842 20216 raft_consensus.cc:348] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.614149 20216 raft_consensus.cc:374] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.614307 20218 ts_tablet_manager.cc:1428] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260619 06:50:30.614311 20216 raft_consensus.cc:729] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a866bcf10bad4c86852bcc8f8d6b9176, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.614713 20216 consensus_queue.cc:260] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.615399 20216 ts_tablet_manager.cc:1428] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260619 06:50:30.618666 20241 ts_tablet_manager.cc:1918] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:30.618898 20241 log.cc:1196] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/5ede1b5f0e2c43889340d14b6610e25c
I20260619 06:50:30.619500 20241 ts_tablet_manager.cc:1939] T 5ede1b5f0e2c43889340d14b6610e25c P 81a2a59deb15495ea9f0cf9fe7705929: Deleting consensus metadata
I20260619 06:50:30.621474 19699 catalog_manager.cc:4841] TS 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709): tablet 5ede1b5f0e2c43889340d14b6610e25c (table test-table [id=b085c4dc1b7147fe8ea6832223290348]) successfully deleted
I20260619 06:50:30.621881 20219 ts_tablet_manager.cc:1428] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260619 06:50:30.627511 20224 raft_consensus.cc:482] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:30.627743 20224 raft_consensus.cc:504] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.629730 20224 leader_election.cc:288] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66:39437), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.637854 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.630680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46257 (local address 127.18.39.67:44709)
0619 06:50:30.630847 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.630854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.631460 (+ 606us) server_negotiation.cc:407] Connection header received
0619 06:50:30.631731 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.631736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.631809 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.631886 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.632780 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.633985 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.636064 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.636837 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.637029 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.637110 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.637455 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.637579 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:50:30.637939 20246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.630623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44709 (local address 127.18.39.65:46257)
0619 06:50:30.631359 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.631381 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.631566 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.632100 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.632104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.632130 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.632548 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.632567 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.634135 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.634141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.635906 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.635918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.637082 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.637088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.637240 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.637264 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.637430 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.637566 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":3,"thread_start_us":403,"threads_started":1}
I20260619 06:50:30.638811 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98332db5eaa7403f845abe5de3048e90" candidate_uuid: "4f65e692de82440e886d62610dd6ba0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929" is_pre_election: true
I20260619 06:50:30.639072 20148 raft_consensus.cc:2454] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f65e692de82440e886d62610dd6ba0b in term 0.
I20260619 06:50:30.639578 19802 leader_election.cc:302] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f65e692de82440e886d62610dd6ba0b, 81a2a59deb15495ea9f0cf9fe7705929; no voters:
I20260619 06:50:30.640010 20224 raft_consensus.cc:2781] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:30.640157 20224 raft_consensus.cc:482] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:30.640261 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.630187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:50839 (local address 127.18.39.66:39437)
0619 06:50:30.630443 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.630499 (+ 56us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.632938 (+ 2439us) server_negotiation.cc:407] Connection header received
0619 06:50:30.633110 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.633119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.633180 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.633281 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:30.633827 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.635022 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.637625 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.638985 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.639115 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.639197 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.639788 (+ 591us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.639923 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:50:30.640280 20224 raft_consensus.cc:3037] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.640427 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.631367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39437 (local address 127.18.39.65:50839)
0619 06:50:30.632834 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.632848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.632984 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.633299 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.633303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.633326 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:30.633676 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.633684 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.635201 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.635207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.637439 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.637456 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.639392 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.639400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.639597 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.639632 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.639735 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.639874 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1347,"thread_start_us":1137,"threads_started":1}
I20260619 06:50:30.641422 20009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98332db5eaa7403f845abe5de3048e90" candidate_uuid: "4f65e692de82440e886d62610dd6ba0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" is_pre_election: true
I20260619 06:50:30.641908 20009 raft_consensus.cc:2454] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f65e692de82440e886d62610dd6ba0b in term 0.
I20260619 06:50:30.643165 20224 raft_consensus.cc:504] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.643882 20224 leader_election.cc:288] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [CANDIDATE]: Term 1 election: Requested vote from peers a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66:39437), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.644419 20009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98332db5eaa7403f845abe5de3048e90" candidate_uuid: "4f65e692de82440e886d62610dd6ba0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a866bcf10bad4c86852bcc8f8d6b9176"
I20260619 06:50:30.644461 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98332db5eaa7403f845abe5de3048e90" candidate_uuid: "4f65e692de82440e886d62610dd6ba0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929"
I20260619 06:50:30.644642 20009 raft_consensus.cc:3037] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.644676 20148 raft_consensus.cc:3037] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.646972 20148 raft_consensus.cc:2454] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f65e692de82440e886d62610dd6ba0b in term 1.
I20260619 06:50:30.647346 19802 leader_election.cc:302] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f65e692de82440e886d62610dd6ba0b, 81a2a59deb15495ea9f0cf9fe7705929; no voters:
I20260619 06:50:30.647645 20224 raft_consensus.cc:2781] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:30.647964 20009 raft_consensus.cc:2454] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f65e692de82440e886d62610dd6ba0b in term 1.
I20260619 06:50:30.648118 20224 raft_consensus.cc:686] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 1 LEADER]: Becoming Leader. State: Replica: 4f65e692de82440e886d62610dd6ba0b, State: Running, Role: LEADER
I20260619 06:50:30.648545 20224 consensus_queue.cc:237] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.652086 19712 catalog_manager.cc:5495] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f65e692de82440e886d62610dd6ba0b (127.18.39.65). New cstate: current_term: 1 leader_uuid: "4f65e692de82440e886d62610dd6ba0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } health_report { overall_health: UNKNOWN } } }
Jun 19 06:50:30 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851828, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:30.707298 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.692032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40645 (local address 127.0.0.1:56834)
0619 06:50:30.692718 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.692742 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.692902 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.693320 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.693323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.694073 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:30.694352 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.694361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.695800 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.695803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.696832 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.696843 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.697899 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.697904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.698031 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.698750 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:30.698765 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:30.701097 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:30.704808 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.704819 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.704825 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.705231 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.705667 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:30.705670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:30.705672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:30.705809 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:30.706435 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:30.706443 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:30.706602 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.706741 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.706949 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":382,"threads_started":1}
I20260619 06:50:30.707578 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.692020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.18.39.126:40645)
0619 06:50:30.692257 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.692265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.692846 (+ 581us) server_negotiation.cc:407] Connection header received
0619 06:50:30.693023 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.693029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.693108 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.693288 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:30.694517 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.695665 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.696975 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.697662 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.697812 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.701297 (+ 3485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:30.701304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:30.701312 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:30.701373 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:30.704675 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.705379 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.705384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.705388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.705544 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:30.705916 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:30.705921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:30.705924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:30.706294 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:30.706450 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.706920 (+ 470us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.707171 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:50:30.713260 19712 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:56834:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:50:30.714605 19712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:30.721614 19848 tablet_service.cc:1467] Processing CreateTablet for tablet 8fac70b3438a4b1284c4853a682e4023 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff5064b68b246699dedef3f89e2c7b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:30.721774 19989 tablet_service.cc:1467] Processing CreateTablet for tablet 8fac70b3438a4b1284c4853a682e4023 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff5064b68b246699dedef3f89e2c7b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:30.722177 19848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fac70b3438a4b1284c4853a682e4023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.722332 19989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fac70b3438a4b1284c4853a682e4023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.727804 20216 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: Bootstrap starting.
I20260619 06:50:30.729142 20219 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: Bootstrap starting.
I20260619 06:50:30.729673 20216 tablet_bootstrap.cc:654] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.730012 20128 tablet_service.cc:1467] Processing CreateTablet for tablet 8fac70b3438a4b1284c4853a682e4023 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff5064b68b246699dedef3f89e2c7b0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:30.730548 20128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fac70b3438a4b1284c4853a682e4023. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:30.730866 20219 tablet_bootstrap.cc:654] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.740516 20216 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: No bootstrap required, opened a new log
I20260619 06:50:30.740814 20216 ts_tablet_manager.cc:1397] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260619 06:50:30.741907 20216 raft_consensus.cc:348] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.742394 20216 raft_consensus.cc:374] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.742568 20216 raft_consensus.cc:729] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a866bcf10bad4c86852bcc8f8d6b9176, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.743045 20216 consensus_queue.cc:260] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.743683 20222 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.743980 20222 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.744088 20216 ts_tablet_manager.cc:1428] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:30.747539 20218 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: Bootstrap starting.
I20260619 06:50:30.749260 20218 tablet_bootstrap.cc:654] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:30.763007 20219 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: No bootstrap required, opened a new log
I20260619 06:50:30.763338 20218 tablet_bootstrap.cc:492] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: No bootstrap required, opened a new log
I20260619 06:50:30.763514 20218 ts_tablet_manager.cc:1397] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.011s
I20260619 06:50:30.764268 20218 raft_consensus.cc:348] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.764422 20219 ts_tablet_manager.cc:1397] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: Time spent bootstrapping tablet: real 0.034s user 0.013s sys 0.012s
I20260619 06:50:30.764523 20218 raft_consensus.cc:374] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.764597 20218 raft_consensus.cc:729] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a2a59deb15495ea9f0cf9fe7705929, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.764882 20218 consensus_queue.cc:260] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.765306 20223 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.765607 20223 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.765529 20219 raft_consensus.cc:348] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.765947 20219 raft_consensus.cc:374] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:30.766057 20219 raft_consensus.cc:729] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f65e692de82440e886d62610dd6ba0b, State: Initialized, Role: FOLLOWER
I20260619 06:50:30.766438 20219 consensus_queue.cc:260] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.766911 20224 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.767187 20224 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.767781 20219 ts_tablet_manager.cc:1428] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:30.768875 20218 ts_tablet_manager.cc:1428] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260619 06:50:30.874675 20222 raft_consensus.cc:482] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:30.874915 20222 raft_consensus.cc:504] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.875698 20222 leader_election.cc:288] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.876264 19874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fac70b3438a4b1284c4853a682e4023" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f65e692de82440e886d62610dd6ba0b" is_pre_election: true
I20260619 06:50:30.876299 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fac70b3438a4b1284c4853a682e4023" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929" is_pre_election: true
I20260619 06:50:30.876546 19874 raft_consensus.cc:2454] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 0.
I20260619 06:50:30.876556 20148 raft_consensus.cc:2454] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 0.
I20260619 06:50:30.876994 19942 leader_election.cc:302] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81a2a59deb15495ea9f0cf9fe7705929, a866bcf10bad4c86852bcc8f8d6b9176; no voters:
I20260619 06:50:30.877279 20222 raft_consensus.cc:2781] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:30.877444 20222 raft_consensus.cc:482] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:30.877558 20222 raft_consensus.cc:3037] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.880856 20222 raft_consensus.cc:504] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.881601 20222 leader_election.cc:288] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 election: Requested vote from peers 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177), 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709)
I20260619 06:50:30.882156 19874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fac70b3438a4b1284c4853a682e4023" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f65e692de82440e886d62610dd6ba0b"
I20260619 06:50:30.882241 20148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fac70b3438a4b1284c4853a682e4023" candidate_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81a2a59deb15495ea9f0cf9fe7705929"
I20260619 06:50:30.882381 19874 raft_consensus.cc:3037] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.882508 20148 raft_consensus.cc:3037] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:30.884877 19874 raft_consensus.cc:2454] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 1.
I20260619 06:50:30.885418 19941 leader_election.cc:302] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f65e692de82440e886d62610dd6ba0b, a866bcf10bad4c86852bcc8f8d6b9176; no voters:
I20260619 06:50:30.885571 20148 raft_consensus.cc:2454] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a866bcf10bad4c86852bcc8f8d6b9176 in term 1.
I20260619 06:50:30.885771 20222 raft_consensus.cc:2781] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:30.886125 20222 raft_consensus.cc:686] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 LEADER]: Becoming Leader. State: Replica: a866bcf10bad4c86852bcc8f8d6b9176, State: Running, Role: LEADER
I20260619 06:50:30.886508 20222 consensus_queue.cc:237] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } }
I20260619 06:50:30.888298 20235 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: TxnStatusTablet state changed. Reason: New leader a866bcf10bad4c86852bcc8f8d6b9176. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.888700 20235 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:30.889665 20260 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:50:30.889925 20260 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:50:30.890079 19712 catalog_manager.cc:5495] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 reported cstate change: term changed from 0 to 1, leader changed from <none> to a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } health_report { overall_health: UNKNOWN } } }
I20260619 06:50:30.936609 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.931446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45200 (local address 127.18.39.66:39437)
0619 06:50:30.931614 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.931620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.931666 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:50:30.931961 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.931971 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.932071 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.932176 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:30.932548 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.933390 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.934715 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.935203 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.935337 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.935815 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.935941 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.936052 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.936202 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.936288 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:50:30.936640 20255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.931347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39437 (local address 127.0.0.1:45200)
0619 06:50:30.931586 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.931607 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.931764 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.932202 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.932207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.932219 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:30.932414 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.932422 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.933549 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.933555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.934567 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.934578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.935386 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.935395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.935556 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.935615 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.936048 (+ 433us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.936074 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.936179 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.936338 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 06:50:30.940922 19874 raft_consensus.cc:1261] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Refusing update from remote peer a866bcf10bad4c86852bcc8f8d6b9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.940974 20148 raft_consensus.cc:1261] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Refusing update from remote peer a866bcf10bad4c86852bcc8f8d6b9176: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.941555 20222 consensus_queue.cc:1035] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:30.941831 20235 consensus_queue.cc:1035] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:30.944592 20224 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: TxnStatusTablet state changed. Reason: New leader a866bcf10bad4c86852bcc8f8d6b9176. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.944916 20224 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.947008 20222 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.947278 20222 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:30.949474 20230 mvcc.cc:205] Tried to move back new op lower bound from 7298465099527180288 to 7298465099313455104. Current Snapshot: MvccSnapshot[applied={T|T < 7298465099527180288 or (T in {7298465099527180288})}]
I20260619 06:50:30.951659 20224 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.951916 20224 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 4f65e692de82440e886d62610dd6ba0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.957805 20227 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.957893 20223 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: TxnStatusTablet state changed. Reason: New leader a866bcf10bad4c86852bcc8f8d6b9176. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.958160 20223 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.958159 20227 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P a866bcf10bad4c86852bcc8f8d6b9176: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:30.961898 20223 tablet_replica.cc:439] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f65e692de82440e886d62610dd6ba0b" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35177 } } peers { permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 } } peers { permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 } } }
I20260619 06:50:30.962219 20223 tablet_replica.cc:442] T 8fac70b3438a4b1284c4853a682e4023 P 81a2a59deb15495ea9f0cf9fe7705929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:30.966953 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.960729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.18.39.65:35177)
0619 06:50:30.960902 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.960907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.960921 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:50:30.961004 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.961008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.961080 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.961275 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:30.961911 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.962927 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.964508 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.964926 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.965081 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.965633 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.966257 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.966422 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.966502 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.966587 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:50:30.984586 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.978275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:57747 (local address 127.18.39.66:39437)
0619 06:50:30.978491 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:30.978497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:30.978879 (+ 382us) server_negotiation.cc:407] Connection header received
0619 06:50:30.979054 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:30.979060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:30.979154 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:30.979275 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:30.979726 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.980913 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.982398 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.983177 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.983324 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.983792 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.983934 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.984043 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:30.984215 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 06:50:30.984306 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:50:30.984624 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:30.978153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39437 (local address 127.18.39.65:57747)
0619 06:50:30.978759 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:30.978776 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:30.978913 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:30.979289 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:30.979292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:30.979301 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:30.979567 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.979581 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.981053 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.981057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.982227 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:30.982240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:30.983401 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:30.983405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:30.983520 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:30.983582 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:30.984065 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:30.984079 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 06:50:30.984161 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 06:50:30.984279 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":318,"threads_started":1}
I20260619 06:50:30.994172 20148 raft_consensus.cc:1261] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Refusing update from remote peer 4f65e692de82440e886d62610dd6ba0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.994649 20009 raft_consensus.cc:1261] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Refusing update from remote peer 4f65e692de82440e886d62610dd6ba0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:30.995456 20248 consensus_queue.cc:1035] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "81a2a59deb15495ea9f0cf9fe7705929" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:30.995831 20224 consensus_queue.cc:1035] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a866bcf10bad4c86852bcc8f8d6b9176" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:31.039597 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.027554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:51833 (local address 127.18.39.65:35177)
0619 06:50:31.027764 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.027770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.034031 (+ 6261us) server_negotiation.cc:407] Connection header received
0619 06:50:31.034229 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.034233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.034311 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.034388 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:31.034934 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.035710 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.037698 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.038346 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.038472 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.038886 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:31.039012 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:31.039103 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.039212 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.039292 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 06:50:31.039623 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.027480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:35177 (local address 127.18.39.66:51833)
0619 06:50:31.033902 (+ 6422us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:31.033924 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:31.034091 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:31.034427 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:31.034432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:31.034444 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:31.034781 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:31.034793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.035875 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.035880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:31.037545 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:31.037561 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.038525 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.038530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:31.038667 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.038722 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:31.039111 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:31.039119 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:50:31.039178 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 06:50:31.039282 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6213,"thread_start_us":6037,"threads_started":1}
I20260619 06:50:31.082489 19712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:31.082767 19712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56822:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:31.086041 19712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 020f7cf4e53046aab455516afe25dca4: Sending DeleteTablet for 3 replicas of tablet 98332db5eaa7403f845abe5de3048e90
I20260619 06:50:31.087123 20128 tablet_service.cc:1514] Processing DeleteTablet for tablet 98332db5eaa7403f845abe5de3048e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:31 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:44616
I20260619 06:50:31.087218 19848 tablet_service.cc:1514] Processing DeleteTablet for tablet 98332db5eaa7403f845abe5de3048e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:31 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53956
I20260619 06:50:31.087440 20241 tablet_replica.cc:330] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: stopping tablet replica
I20260619 06:50:31.087473 20242 tablet_replica.cc:330] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: stopping tablet replica
I20260619 06:50:31.087648 20242 raft_consensus.cc:2229] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:31.087632 20241 raft_consensus.cc:2229] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:31.087829 20241 raft_consensus.cc:2258] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:31.088200 20242 raft_consensus.cc:2258] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:31.089154 19989 tablet_service.cc:1514] Processing DeleteTablet for tablet 98332db5eaa7403f845abe5de3048e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:31 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:45168
I20260619 06:50:31.089486 20241 ts_tablet_manager.cc:1905] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:31.090065 20242 ts_tablet_manager.cc:1905] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:31.096764 20241 ts_tablet_manager.cc:1918] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:31.096922 20241 log.cc:1196] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/98332db5eaa7403f845abe5de3048e90
I20260619 06:50:31.099692 20241 ts_tablet_manager.cc:1939] T 98332db5eaa7403f845abe5de3048e90 P 81a2a59deb15495ea9f0cf9fe7705929: Deleting consensus metadata
I20260619 06:50:31.100867 19699 catalog_manager.cc:4841] TS 81a2a59deb15495ea9f0cf9fe7705929 (127.18.39.67:44709): tablet 98332db5eaa7403f845abe5de3048e90 (table test-table [id=fc00d218e5194460a8efb5cef362923c]) successfully deleted
I20260619 06:50:31.102530 20242 ts_tablet_manager.cc:1918] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:31.103461 20242 log.cc:1196] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/98332db5eaa7403f845abe5de3048e90
I20260619 06:50:31.104224 20242 ts_tablet_manager.cc:1939] T 98332db5eaa7403f845abe5de3048e90 P 4f65e692de82440e886d62610dd6ba0b: Deleting consensus metadata
I20260619 06:50:31.106629 19698 catalog_manager.cc:4841] TS 4f65e692de82440e886d62610dd6ba0b (127.18.39.65:35177): tablet 98332db5eaa7403f845abe5de3048e90 (table test-table [id=fc00d218e5194460a8efb5cef362923c]) successfully deleted
I20260619 06:50:31.112243 20274 tablet_replica.cc:330] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: stopping tablet replica
I20260619 06:50:31.112672 20274 raft_consensus.cc:2229] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:31.113126 20274 raft_consensus.cc:2258] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:31.114739 20274 ts_tablet_manager.cc:1905] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:31.121501 20274 ts_tablet_manager.cc:1918] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:31.121646 20274 log.cc:1196] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/98332db5eaa7403f845abe5de3048e90
I20260619 06:50:31.122108 20274 ts_tablet_manager.cc:1939] T 98332db5eaa7403f845abe5de3048e90 P a866bcf10bad4c86852bcc8f8d6b9176: Deleting consensus metadata
I20260619 06:50:31.123231 19698 catalog_manager.cc:4841] TS a866bcf10bad4c86852bcc8f8d6b9176 (127.18.39.66:39437): tablet 98332db5eaa7403f845abe5de3048e90 (table test-table [id=fc00d218e5194460a8efb5cef362923c]) successfully deleted
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:31.134425 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.106787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.18.39.65:35177)
0619 06:50:31.106999 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.107004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.114086 (+ 7082us) server_negotiation.cc:407] Connection header received
0619 06:50:31.117379 (+ 3293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.117384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.117472 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.117551 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:31.119807 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.121025 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.123137 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.123814 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.123934 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.128201 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:31.128215 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:31.128236 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:31.128269 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:31.131644 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.132298 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.132308 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.132311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.132459 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.132844 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.132849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.132853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.133244 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:31.133629 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.133844 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.134094 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 06:50:31.135344 19914 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53972
I20260619 06:50:31.151960 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.138935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.18.39.126:40645)
0619 06:50:31.139175 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.139184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.139621 (+ 437us) server_negotiation.cc:407] Connection header received
0619 06:50:31.139807 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.139813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.139888 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.139992 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:31.141433 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.142167 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.143531 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.144009 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.144145 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.146589 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:31.146596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:31.146600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:31.146643 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:31.149289 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.149992 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.149999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.150002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.150130 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.150522 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.150529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.150532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.150806 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:31.151055 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.151396 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.151623 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260619 06:50:31.152709 19712 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56846
I20260619 06:50:31.154975 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19783
I20260619 06:50:31.168998 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.156783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.18.39.65:35177)
0619 06:50:31.156956 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.156962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.157161 (+ 199us) server_negotiation.cc:407] Connection header received
0619 06:50:31.157406 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.157412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.157480 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.157584 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:31.158643 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.159362 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.160545 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.161001 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.161137 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.163313 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:31.163320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:31.163324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:31.163367 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:31.166228 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.166893 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.166900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.166904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.167031 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.167477 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.167484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.167487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.167843 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:31.168097 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.168376 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.168608 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 06:50:31.169713 19914 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53978
W20260619 06:50:31.170518 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19783: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:31.170722 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19783
I20260619 06:50:31.195399 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19923
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:31.211971 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.196229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.18.39.66:39437)
0619 06:50:31.196440 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.196448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.196476 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:50:31.196733 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.196743 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.196811 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.196939 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:31.198233 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.198996 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.200271 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.200722 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.200876 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.204947 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:31.204962 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:31.204978 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:31.205010 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:31.208762 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.209639 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.209650 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.209654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.209806 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.210293 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.210298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.210302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.210772 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:31.211111 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.211353 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.211605 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260619 06:50:31.212709 20054 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45214
W20260619 06:50:31.213430 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19923: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:31.213617 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19923
I20260619 06:50:31.235411 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20063
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851830, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:31.251183 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:31.236249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.18.39.67:44709)
0619 06:50:31.236767 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:31.236772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:31.236789 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:50:31.236842 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:31.236845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:31.236914 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:31.237003 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:31.238275 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.239015 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.240168 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:31.240567 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:31.240658 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:31.244844 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:31.244862 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:31.244879 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:31.244915 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:31.248194 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.248840 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.248848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.248852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.248985 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:31.249318 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:31.249323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:31.249325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:31.250058 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:31.250337 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:31.250568 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 06:50:31.250798 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":283,"threads_started":1}
W20260619 06:50:31.251974 20193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44630
W20260619 06:50:31.252687 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20063: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:31.252822 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 20063
I20260619 06:50:31.269030 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 19680
I20260619 06:50:31.270260 19752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:31.385515 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 19680
2026-06-19T06:50:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4527 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[20284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[20284](info): set up 2 sockets
Jun 19 06:50:31 dist-test-slave-1h5s krb5kdc[20284](info): commencing operation
krb5kdc: starting...
W20260619 06:50:32.469605 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:32 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:32.654882 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40955
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:32.977275 20297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:32.977612 20297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:32.977782 20297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:32.985049 20297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:32.985162 20297 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:50:32.985205 20297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:32.985320 20297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:32.985412 20297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:32.985502 20297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:32.995700 20297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40955
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:32.997098 20297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:32.998522 20297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:33.010775 20302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:33.010741 20305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:33.010696 20303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:33.011910 20297 server_base.cc:1034] running on GCE node
I20260619 06:50:33.012691 20297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:33.014477 20297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:33.015753 20297 hybrid_clock.cc:648] HybridClock initialized: now 1781851833015665 us; error 64 us; skew 500 ppm
I20260619 06:50:33.016124 20297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:33 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851833, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:33.019428 20297 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:33.020946 20297 webserver.cc:460] Webserver started at http://127.18.39.126:45391/ using document root <none> and password file <none>
I20260619 06:50:33.021770 20297 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:33.021889 20297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:33.022224 20297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:33.025177 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "efe747fed9314d5fbef46070bef26601"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "39671e6cc126a4e7ed2b7ba71ba31198"
server_key_iv: "b70d03139d3bb6172c17e48f1d15f54a"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.026036 20297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "efe747fed9314d5fbef46070bef26601"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "39671e6cc126a4e7ed2b7ba71ba31198"
server_key_iv: "b70d03139d3bb6172c17e48f1d15f54a"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.031304 20297 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:50:33.034919 20312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:33.036243 20297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:50:33.036437 20297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "efe747fed9314d5fbef46070bef26601"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "39671e6cc126a4e7ed2b7ba71ba31198"
server_key_iv: "b70d03139d3bb6172c17e48f1d15f54a"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.036669 20297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:33.063618 20297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:33.067850 20297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:33.068177 20297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:33.090639 20297 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:40955
I20260619 06:50:33.090658 20373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:40955 every 8 connection(s)
I20260619 06:50:33.092022 20297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:33.093443 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 20297
I20260619 06:50:33.093641 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:33.094445 18589 external_mini_cluster.cc:1428] Setting key 134d3446eb0c8ecdc701518d31893bb2
I20260619 06:50:33.096901 20374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:50:33 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851832, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:33.110481 20374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Bootstrap starting.
I20260619 06:50:33.114866 20374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:33.115948 20374 log.cc:824] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:33.118456 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:33.096508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.18.39.126:40955)
0619 06:50:33.097302 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:33.097313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:33.097407 (+ 94us) server_negotiation.cc:407] Connection header received
0619 06:50:33.098974 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:33.098995 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:33.099383 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:33.099717 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:33.101518 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.102899 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.104635 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.105327 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.105537 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:33.110301 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:33.110313 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:33.110335 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:33.110392 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:33.114324 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.115082 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.115090 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.115096 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.115285 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.115661 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.115668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.115672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.116268 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:33.116609 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:33.117050 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 06:50:33.117495 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":393,"threads_started":1}
I20260619 06:50:33.119014 20374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: No bootstrap required, opened a new log
I20260619 06:50:33.126691 20374 raft_consensus.cc:348] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } }
I20260619 06:50:33.127091 20374 raft_consensus.cc:374] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:33.127200 20374 raft_consensus.cc:729] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe747fed9314d5fbef46070bef26601, State: Initialized, Role: FOLLOWER
I20260619 06:50:33.127863 20374 consensus_queue.cc:260] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } }
I20260619 06:50:33.128136 20374 raft_consensus.cc:388] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:33.128284 20374 raft_consensus.cc:482] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:33.128468 20374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:33.130752 20374 raft_consensus.cc:504] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } }
I20260619 06:50:33.131238 20374 leader_election.cc:302] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efe747fed9314d5fbef46070bef26601; no voters:
I20260619 06:50:33.131803 20374 leader_election.cc:288] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:33.131958 20379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:33.132634 20379 raft_consensus.cc:686] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [term 1 LEADER]: Becoming Leader. State: Replica: efe747fed9314d5fbef46070bef26601, State: Running, Role: LEADER
I20260619 06:50:33.133141 20379 consensus_queue.cc:237] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } }
I20260619 06:50:33.133966 20374 sys_catalog.cc:564] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:33.135808 20381 sys_catalog.cc:455] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efe747fed9314d5fbef46070bef26601. Latest consensus state: current_term: 1 leader_uuid: "efe747fed9314d5fbef46070bef26601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } } }
I20260619 06:50:33.136157 20381 sys_catalog.cc:458] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:33.136921 20380 sys_catalog.cc:455] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efe747fed9314d5fbef46070bef26601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe747fed9314d5fbef46070bef26601" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40955 } } }
I20260619 06:50:33.137372 20380 sys_catalog.cc:458] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:33.138497 20384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:33.143757 20384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:33.153450 20384 catalog_manager.cc:1347] Generated new cluster ID: 088ffca6a7f1447dbdc3371494a61cbc
I20260619 06:50:33.153576 20384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:33.163074 20384 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:33.164918 20384 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:33.191286 20384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Generated new TSK 0
I20260619 06:50:33.192461 20384 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:33.245939 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40955
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:33.579309 20400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:33.579687 20400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:33.579871 20400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:33.587387 20400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:33.587584 20400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:33.587741 20400 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:50:33.587829 20400 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:33.587968 20400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:33.598268 20400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:33.599686 20400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:33.601059 20400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:33.615425 20408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:33.615521 20406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:33.615541 20405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:33.616667 20400 server_base.cc:1034] running on GCE node
I20260619 06:50:33.617400 20400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:33.618551 20400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:33.619799 20400 hybrid_clock.cc:648] HybridClock initialized: now 1781851833619754 us; error 45 us; skew 500 ppm
I20260619 06:50:33.620123 20400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:33 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851833, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:33.623674 20400 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:33.625164 20400 webserver.cc:460] Webserver started at http://127.18.39.65:45287/ using document root <none> and password file <none>
I20260619 06:50:33.625908 20400 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:33.626014 20400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:33.626269 20400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:33.629213 20400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "ab03559e33d1412ab413c145c960345f"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "c054e2d9dc54e01f087f8e39db47729f"
server_key_iv: "4d7080f05e9e7d4fec343baaedafc538"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.630028 20400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "ab03559e33d1412ab413c145c960345f"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "c054e2d9dc54e01f087f8e39db47729f"
server_key_iv: "4d7080f05e9e7d4fec343baaedafc538"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.635452 20400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:33.639025 20415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:33.640650 20400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 06:50:33.640812 20400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "ab03559e33d1412ab413c145c960345f"
format_stamp: "Formatted at 2026-06-19 06:50:33 on dist-test-slave-1h5s"
server_key: "c054e2d9dc54e01f087f8e39db47729f"
server_key_iv: "4d7080f05e9e7d4fec343baaedafc538"
server_key_version: "encryptionkey@0"
I20260619 06:50:33.641002 20400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:33.657634 20400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:33.662626 20400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:33.662974 20400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:33.665740 20400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:33.665867 20400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:33.665983 20400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:33.666069 20400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:50:33 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851833, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:33.694983 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:33.666396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:37125 (local address 127.18.39.126:40955)
0619 06:50:33.667007 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:33.667012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:33.668282 (+ 1270us) server_negotiation.cc:407] Connection header received
0619 06:50:33.670463 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:33.670469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:33.670581 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:33.670705 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:33.673123 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.674392 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.678061 (+ 3669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.678843 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.679009 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:33.685651 (+ 6642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:33.685663 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:33.685670 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:33.685721 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:33.689684 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.691199 (+ 1515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.691208 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.691214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.691401 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.692095 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.692100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.692104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.692557 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:33.692727 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:33.693969 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0619 06:50:33.694355 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":390,"threads_started":1}
I20260619 06:50:33.695672 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:33.667206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.65:37125)
0619 06:50:33.668103 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:33.668144 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:33.670148 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:33.671027 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:33.671045 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:33.671951 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:33.672877 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:33.672907 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.674609 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.674617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:33.677819 (+ 3202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:33.677846 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.679173 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.679200 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:33.679662 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:33.680754 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:33.680800 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:33.685232 (+ 4432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:33.689926 (+ 4694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:33.689944 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:33.689970 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:33.690995 (+ 1025us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:33.691583 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:33.691591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:33.691596 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:33.691947 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:33.692713 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:33.692723 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:33.693391 (+ 668us) client_negotiation.cc:769] Sending connection context
0619 06:50:33.693818 (+ 427us) client_negotiation.cc:240] Negotiation successful
0619 06:50:33.694349 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":372,"threads_started":1}
I20260619 06:50:33.710498 20400 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41235
I20260619 06:50:33.710520 20535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41235 every 8 connection(s)
I20260619 06:50:33.712365 20400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:33.714764 20536 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40955
I20260619 06:50:33.715096 20536 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:33.715981 20536 heartbeater.cc:511] Master 127.18.39.126:40955 requested a full tablet report, sending...
I20260619 06:50:33.716535 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 20400
I20260619 06:50:33.716761 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:33.717558 18589 external_mini_cluster.cc:1428] Setting key ea7ec8f3f67eca352255a413f16d58b5
I20260619 06:50:33.718704 20329 ts_manager.cc:194] Registered new tserver with Master: ab03559e33d1412ab413c145c960345f (127.18.39.65:41235)
I20260619 06:50:33.718886 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:33.702921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46495 (local address 127.18.39.126:40955)
0619 06:50:33.703161 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:33.703169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:33.704227 (+ 1058us) server_negotiation.cc:407] Connection header received
0619 06:50:33.704487 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:33.704491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:33.704557 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:33.704679 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:33.706201 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.707241 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.709109 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.709684 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.709824 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:33.712393 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:33.712398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:33.712401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:33.712432 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:33.715569 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.716518 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.716526 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.716530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.716702 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:33.717121 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:33.717126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:33.717129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:33.717615 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:33.717813 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:33.718307 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 06:50:33.718548 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:50:33.719066 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:33.702928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.65:46495)
0619 06:50:33.704081 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:33.704107 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:33.704344 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:33.704709 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:33.704714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:33.705523 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:33.706054 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:33.706066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.707397 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.707402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:33.708962 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:33.708974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:33.709893 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:33.709905 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:33.710192 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:33.710986 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:33.711010 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:33.712175 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:33.715701 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:33.715707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:33.715710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:33.716371 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:33.716834 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:33.716840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:33.716843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:33.717001 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:33.717803 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:33.717809 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:33.718018 (+ 209us) client_negotiation.cc:769] Sending connection context
0619 06:50:33.718206 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 06:50:33.718559 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"thread_start_us":733,"threads_started":1}
I20260619 06:50:33.720767 20329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:37125
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:33.761237 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40955
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:34.090585 20540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:34.090971 20540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:34.091202 20540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:34.103005 20540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:34.103152 20540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:34.103287 20540 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:50:34.103356 20540 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:34.103454 20540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:34.117300 20540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:34.118677 20540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:34.120025 20540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:34.133844 20546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:34.133867 20548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:34.133922 20545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:34.135125 20540 server_base.cc:1034] running on GCE node
I20260619 06:50:34.135787 20540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:34.136780 20540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:34.138021 20540 hybrid_clock.cc:648] HybridClock initialized: now 1781851834137980 us; error 71 us; skew 500 ppm
I20260619 06:50:34.138350 20540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:34.141826 20540 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:34.143296 20540 webserver.cc:460] Webserver started at http://127.18.39.66:36219/ using document root <none> and password file <none>
I20260619 06:50:34.144014 20540 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:34.144163 20540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:34.144480 20540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:34.147809 20540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "dcfa021d06264f368f49a9ee88eb825f"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "46cc1475b3a3b208cfad73dc70681c15"
server_key_iv: "073ea6a00ed5a124f68241dffbb68b9d"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.148590 20540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "dcfa021d06264f368f49a9ee88eb825f"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "46cc1475b3a3b208cfad73dc70681c15"
server_key_iv: "073ea6a00ed5a124f68241dffbb68b9d"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.154062 20540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:50:34.157572 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:34.158901 20540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:50:34.159096 20540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "dcfa021d06264f368f49a9ee88eb825f"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "46cc1475b3a3b208cfad73dc70681c15"
server_key_iv: "073ea6a00ed5a124f68241dffbb68b9d"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.159307 20540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:34.183161 20540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:34.187469 20540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:34.187793 20540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:34.190713 20540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:34.190862 20540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:34.191013 20540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:34.191128 20540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:34.216820 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.191202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:38777 (local address 127.18.39.126:40955)
0619 06:50:34.191392 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.191399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.192957 (+ 1558us) server_negotiation.cc:407] Connection header received
0619 06:50:34.195994 (+ 3037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.196000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.196092 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.196234 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:34.198955 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.200089 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.202531 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.203100 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.203272 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.209384 (+ 6112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:34.209393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:34.209402 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:34.209448 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:34.213135 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.214232 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.214238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.214242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.214433 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.215007 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.215013 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.215021 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.215319 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:34.215477 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.216231 (+ 754us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.216467 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 06:50:34.218102 20564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.191904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.66:38777)
0619 06:50:34.192764 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.192818 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.195616 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.196734 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.196754 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.197628 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:34.198720 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.198747 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.200259 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.200264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.202372 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.202385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.203615 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.203623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.203984 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.204858 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:34.204899 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:34.209014 (+ 4115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:34.213312 (+ 4298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.213325 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.213395 (+ 70us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.214064 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.214591 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.214596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.214601 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.214869 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.215454 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:34.215463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:34.215870 (+ 407us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.216137 (+ 267us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.216691 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":323,"threads_started":1}
I20260619 06:50:34.241060 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.225311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.66:38163)
0619 06:50:34.226199 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.226218 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.226403 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.226777 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.226782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.227592 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:34.228089 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.228106 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.229607 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.229612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.231236 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.231247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.232041 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.232047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.232220 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.232928 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:34.232945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:34.234063 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:34.237989 (+ 3926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.237996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.238005 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.238630 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.239130 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.239135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.239139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.239300 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.239994 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:34.239999 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:34.240206 (+ 207us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.240355 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.240632 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":456,"threads_started":1}
I20260619 06:50:34.241909 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.225245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:38163 (local address 127.18.39.126:40955)
0619 06:50:34.225463 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.225470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.226323 (+ 853us) server_negotiation.cc:407] Connection header received
0619 06:50:34.226556 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.226562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.226638 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.226747 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:34.228299 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.229448 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.231396 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.231845 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.231995 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.234246 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:34.234252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:34.234257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:34.234298 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:34.237793 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.238814 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.238817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.238820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.239002 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.239413 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.239420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.239424 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.239860 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:34.240017 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.241427 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.241691 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:50:34.246986 20540 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:39653
I20260619 06:50:34.247010 20674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:39653 every 8 connection(s)
I20260619 06:50:34.248224 20540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:34.249799 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 20540
I20260619 06:50:34.249979 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:34.250619 18589 external_mini_cluster.cc:1428] Setting key 6ce63e5f99899822e58759f65a42363f
I20260619 06:50:34.250372 20675 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40955
I20260619 06:50:34.251616 20675 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:34.252291 20675 heartbeater.cc:511] Master 127.18.39.126:40955 requested a full tablet report, sending...
I20260619 06:50:34.253895 20329 ts_manager.cc:194] Registered new tserver with Master: dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:34.254830 20329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:38777
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:34.292305 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40955
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:34.630191 20679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:34.630537 20679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:34.630709 20679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:34.638170 20679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:34.638350 20679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:34.638495 20679 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:50:34.638582 20679 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:50:34.638710 20679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:34.648777 20679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:34.650151 20679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:34.651526 20679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:34.664656 20684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:34.664707 20685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:34.664736 20687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:34.665990 20679 server_base.cc:1034] running on GCE node
I20260619 06:50:34.666602 20679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:34.667538 20679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:34.668736 20679 hybrid_clock.cc:648] HybridClock initialized: now 1781851834668693 us; error 35 us; skew 500 ppm
I20260619 06:50:34.669046 20679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:34.672102 20679 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:34.673578 20679 webserver.cc:460] Webserver started at http://127.18.39.67:35151/ using document root <none> and password file <none>
I20260619 06:50:34.674201 20679 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:34.674295 20679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:34.674563 20679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:34.677412 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "b2e6bc3101c013ac433b6efa99b09fc1"
server_key_iv: "1bd9f12c6f9b13725c0a6cf6b229a7b4"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.678109 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "b2e6bc3101c013ac433b6efa99b09fc1"
server_key_iv: "1bd9f12c6f9b13725c0a6cf6b229a7b4"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.683071 20679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:50:34.686451 20694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:34.687762 20679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:50:34.687948 20679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
format_stamp: "Formatted at 2026-06-19 06:50:34 on dist-test-slave-1h5s"
server_key: "b2e6bc3101c013ac433b6efa99b09fc1"
server_key_iv: "1bd9f12c6f9b13725c0a6cf6b229a7b4"
server_key_version: "encryptionkey@0"
I20260619 06:50:34.688158 20679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:34.720614 20679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:34.724495 20536 heartbeater.cc:503] Master 127.18.39.126:40955 was elected leader, sending a full tablet report...
I20260619 06:50:34.724851 20679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:34.725196 20679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:34.728111 20679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:34.728235 20679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:34.728371 20679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:34.728470 20679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:34.755192 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.728878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:39857 (local address 127.18.39.126:40955)
0619 06:50:34.729070 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.729076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.730679 (+ 1603us) server_negotiation.cc:407] Connection header received
0619 06:50:34.734040 (+ 3361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.734049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.734146 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.734277 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:34.736745 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.737963 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.741024 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.741981 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.742174 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.747730 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:34.747735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:34.747739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:34.747778 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:34.751384 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.752530 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.752538 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.752544 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.752718 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.753299 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.753307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.753310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.753629 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:34.753782 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.754600 (+ 818us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.754835 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:50:34.756294 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.729562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.67:39857)
0619 06:50:34.730471 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.730533 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.733586 (+ 3053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.734565 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.734574 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.735577 (+ 1003us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:34.736537 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.736563 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.738147 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.738158 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.740798 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.740822 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.742235 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.742240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.742480 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.743359 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:34.743389 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:34.747417 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:34.751582 (+ 4165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.751596 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.751617 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.752290 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.752933 (+ 643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.752938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.752941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.753176 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.753776 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:34.753784 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:34.754192 (+ 408us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.754482 (+ 290us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.755065 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":346,"threads_started":1}
I20260619 06:50:34.782681 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.766562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.18.39.67:34145)
0619 06:50:34.767402 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.767426 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.767623 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.768072 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.768077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.769156 (+ 1079us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:34.769643 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.769682 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.771103 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.771114 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.773010 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.773030 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.773959 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.773967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.774122 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.774835 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:34.774866 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:34.775753 (+ 887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:34.779396 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.779403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.779408 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.780055 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.780597 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:34.780605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:34.780610 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:34.780751 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:34.781465 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:34.781470 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:34.781762 (+ 292us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.781975 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.782338 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":426,"threads_started":1}
I20260619 06:50:34.783442 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.766559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34145 (local address 127.18.39.126:40955)
0619 06:50:34.766805 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.766811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.767562 (+ 751us) server_negotiation.cc:407] Connection header received
0619 06:50:34.767795 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.767802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.767903 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.768045 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:34.769917 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.770879 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.773227 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.773723 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.773865 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.775965 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:34.775971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:34.775974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:34.776007 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:34.779244 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.780238 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.780246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.780253 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.780450 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.780878 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.780892 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.780896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.781330 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:34.781503 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.783024 (+ 1521us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.783209 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:50:34.787832 20679 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:41759
I20260619 06:50:34.787887 20813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:41759 every 8 connection(s)
I20260619 06:50:34.789044 20679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:34.791105 20814 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40955
I20260619 06:50:34.791356 20814 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:34.792047 20814 heartbeater.cc:511] Master 127.18.39.126:40955 requested a full tablet report, sending...
I20260619 06:50:34.792620 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 20679
I20260619 06:50:34.792810 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:34.793334 18589 external_mini_cluster.cc:1428] Setting key 98cc961b2bea3986691144d0b39ab5eb
I20260619 06:50:34.793642 20329 ts_manager.cc:194] Registered new tserver with Master: 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759)
I20260619 06:50:34.794673 20329 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:39857
I20260619 06:50:34.796555 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:50:34 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:34.845062 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.830294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47008 (local address 127.18.39.126:40955)
0619 06:50:34.830558 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.830564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.830751 (+ 187us) server_negotiation.cc:407] Connection header received
0619 06:50:34.830978 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.830984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.831058 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.831184 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:34.832661 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.833473 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.834829 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.835325 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.835476 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.839550 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:34.839555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:34.839558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:34.839595 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:34.842308 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.843000 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.843006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.843015 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.843136 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:34.843537 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:34.843542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:34.843550 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:34.843868 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:34.844129 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.844458 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.844686 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260619 06:50:34.849288 20329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:34.856575 20329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:34.882136 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.869496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.18.39.67:41759)
0619 06:50:34.870267 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.870275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.870310 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:50:34.870465 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.870473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.870732 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.870938 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:34.871842 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.874111 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.877822 (+ 3711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.879924 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.880064 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.880447 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.881414 (+ 967us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.881660 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":407,"threads_started":1}
I20260619 06:50:34.883571 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.869131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41759 (local address 127.0.0.1:53058)
0619 06:50:34.870035 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.870081 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.870244 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.870950 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.870959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.870990 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:34.871514 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.871549 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.874364 (+ 2815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.874373 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.877578 (+ 3205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.877600 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.880683 (+ 3083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.880694 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.881026 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.881101 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.882765 (+ 1664us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.883071 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":983,"threads_started":1}
I20260619 06:50:34.889205 20744 tablet_service.cc:1467] Processing CreateTablet for tablet 697dabe7791f4075b71e772f8ca511c1 (DEFAULT_TABLE table=test-table [id=0c38ef09229b42488f9f4c94d147ee4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:34.890852 20744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697dabe7791f4075b71e772f8ca511c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:34.893389 20827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.871769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39653 (local address 127.0.0.1:60430)
0619 06:50:34.884117 (+ 12348us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.884143 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.884310 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.887748 (+ 3438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.887753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.887780 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:34.888019 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.888031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.889258 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.889263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.890722 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.890735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.892418 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.892424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.892651 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.892683 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.892750 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.892929 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12205,"mutex_wait_us":46,"thread_start_us":12823,"threads_started":1}
I20260619 06:50:34.893386 20829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.870914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.0.0.1:48598)
0619 06:50:34.871726 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:34.871743 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:34.871842 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:34.885889 (+ 14047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:34.885895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:34.885925 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:34.886323 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.886335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.888378 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.888383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.890528 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:34.890545 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.892554 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.892561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:34.892727 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.892759 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:50:34.892858 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:50:34.892998 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":30,"thread_start_us":535,"threads_started":1}
I20260619 06:50:34.893953 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.871171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.18.39.65:41235)
0619 06:50:34.885216 (+ 14045us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.885223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.885253 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:50:34.885401 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.885414 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.885649 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.886525 (+ 876us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:34.886737 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.888232 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.890700 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.892083 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.892212 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.892408 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.893481 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.893644 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13892,"thread_start_us":13774,"threads_started":1}
I20260619 06:50:34.894047 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:34.872229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.18.39.66:39653)
0619 06:50:34.887166 (+ 14937us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:34.887175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:34.887204 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:50:34.887296 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:34.887303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:34.887535 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:34.887725 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:34.888177 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.889099 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.890895 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:34.892227 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:34.893313 (+ 1086us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:34.893535 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:34.893607 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 06:50:34.893763 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14781,"thread_start_us":14672,"threads_started":1}
I20260619 06:50:34.897094 20605 tablet_service.cc:1467] Processing CreateTablet for tablet 697dabe7791f4075b71e772f8ca511c1 (DEFAULT_TABLE table=test-table [id=0c38ef09229b42488f9f4c94d147ee4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:34.898540 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697dabe7791f4075b71e772f8ca511c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:34.900414 20466 tablet_service.cc:1467] Processing CreateTablet for tablet 697dabe7791f4075b71e772f8ca511c1 (DEFAULT_TABLE table=test-table [id=0c38ef09229b42488f9f4c94d147ee4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:34.901726 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 697dabe7791f4075b71e772f8ca511c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:34.903520 20833 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Bootstrap starting.
I20260619 06:50:34.906661 20833 tablet_bootstrap.cc:654] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:34.907701 20833 log.cc:824] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:34.911579 20833 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: No bootstrap required, opened a new log
I20260619 06:50:34.911892 20833 ts_tablet_manager.cc:1397] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260619 06:50:34.912958 20835 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Bootstrap starting.
I20260619 06:50:34.915582 20836 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Bootstrap starting.
I20260619 06:50:34.916774 20835 tablet_bootstrap.cc:654] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:34.918088 20835 log.cc:824] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:34.918717 20836 tablet_bootstrap.cc:654] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:34.920007 20836 log.cc:824] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:34.921417 20835 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: No bootstrap required, opened a new log
I20260619 06:50:34.921702 20835 ts_tablet_manager.cc:1397] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 06:50:34.922358 20833 raft_consensus.cc:348] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.922736 20833 raft_consensus.cc:374] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:34.922834 20833 raft_consensus.cc:729] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Initialized, Role: FOLLOWER
I20260619 06:50:34.923161 20836 tablet_bootstrap.cc:492] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: No bootstrap required, opened a new log
I20260619 06:50:34.923419 20836 ts_tablet_manager.cc:1397] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260619 06:50:34.923465 20833 consensus_queue.cc:260] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.924638 20814 heartbeater.cc:503] Master 127.18.39.126:40955 was elected leader, sending a full tablet report...
I20260619 06:50:34.925298 20833 ts_tablet_manager.cc:1428] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent starting tablet: real 0.013s user 0.003s sys 0.011s
I20260619 06:50:34.932472 20835 raft_consensus.cc:348] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.932941 20835 raft_consensus.cc:374] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:34.933039 20835 raft_consensus.cc:729] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Initialized, Role: FOLLOWER
I20260619 06:50:34.933712 20835 consensus_queue.cc:260] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.933774 20836 raft_consensus.cc:348] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.934242 20836 raft_consensus.cc:374] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:34.934366 20836 raft_consensus.cc:729] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab03559e33d1412ab413c145c960345f, State: Initialized, Role: FOLLOWER
I20260619 06:50:34.934868 20675 heartbeater.cc:503] Master 127.18.39.126:40955 was elected leader, sending a full tablet report...
I20260619 06:50:34.934937 20836 consensus_queue.cc:260] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:34.935679 20835 ts_tablet_manager.cc:1428] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260619 06:50:34.936569 20836 ts_tablet_manager.cc:1428] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
W20260619 06:50:34.964980 20537 tablet.cc:2369] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:35.000308 20676 tablet.cc:2369] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:35.040712 20815 tablet.cc:2369] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:35.084841 20841 raft_consensus.cc:482] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:35.085215 20841 raft_consensus.cc:504] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:35.086952 20841 leader_election.cc:288] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:35.094977 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.087462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42039 (local address 127.18.39.67:41759)
0619 06:50:35.087641 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.087646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.088295 (+ 649us) server_negotiation.cc:407] Connection header received
0619 06:50:35.088629 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.088635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.088711 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.088777 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.089502 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.090430 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.092834 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.094033 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.094203 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.094273 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.094604 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.094729 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:50:35.095113 20842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.087434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41759 (local address 127.18.39.65:42039)
0619 06:50:35.088195 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.088215 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.088426 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.088944 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.088947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.089014 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.089307 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.089315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.090550 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.090558 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.092686 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.092701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.094256 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.094263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.094443 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.094468 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.094576 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.094707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":23,"spinlock_wait_cycles":19200,"thread_start_us":362,"threads_started":1}
I20260619 06:50:35.095512 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.088462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:33023 (local address 127.18.39.66:39653)
0619 06:50:35.088666 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.088673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.088925 (+ 252us) server_negotiation.cc:407] Connection header received
0619 06:50:35.089121 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.089126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.089191 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.089283 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.089894 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.090945 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.093240 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.094076 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.094264 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.094345 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.094798 (+ 453us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.094944 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:50:35.096242 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.088252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39653 (local address 127.18.39.65:33023)
0619 06:50:35.088812 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.088825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.088980 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.089320 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.089326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.089371 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.089724 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.089737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.091133 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.091138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.093104 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.093114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.094442 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.094448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.094617 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.094647 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.095750 (+ 1103us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.095898 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":19,"thread_start_us":293,"threads_started":1}
I20260619 06:50:35.096155 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697dabe7791f4075b71e772f8ca511c1" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" is_pre_election: true
I20260619 06:50:35.096781 20768 raft_consensus.cc:2454] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 0.
I20260619 06:50:35.097316 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697dabe7791f4075b71e772f8ca511c1" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f" is_pre_election: true
I20260619 06:50:35.097498 20417 leader_election.cc:302] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, ab03559e33d1412ab413c145c960345f; no voters:
I20260619 06:50:35.097862 20628 raft_consensus.cc:2454] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 0.
I20260619 06:50:35.097914 20841 raft_consensus.cc:2781] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:35.098052 20841 raft_consensus.cc:482] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:35.098153 20841 raft_consensus.cc:3037] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.100498 20841 raft_consensus.cc:504] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:35.101063 20841 leader_election.cc:288] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 election: Requested vote from peers 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:35.101665 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697dabe7791f4075b71e772f8ca511c1" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
I20260619 06:50:35.101688 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "697dabe7791f4075b71e772f8ca511c1" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f"
I20260619 06:50:35.101872 20768 raft_consensus.cc:3037] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.101871 20628 raft_consensus.cc:3037] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.103961 20768 raft_consensus.cc:2454] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 1.
I20260619 06:50:35.104354 20417 leader_election.cc:302] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, ab03559e33d1412ab413c145c960345f; no voters:
I20260619 06:50:35.104647 20841 raft_consensus.cc:2781] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:35.104924 20628 raft_consensus.cc:2454] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 1.
I20260619 06:50:35.105947 20841 raft_consensus.cc:686] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 1 LEADER]: Becoming Leader. State: Replica: ab03559e33d1412ab413c145c960345f, State: Running, Role: LEADER
I20260619 06:50:35.106511 20841 consensus_queue.cc:237] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } }
I20260619 06:50:35.111299 20329 catalog_manager.cc:5495] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f reported cstate change: term changed from 0 to 1, leader changed from <none> to ab03559e33d1412ab413c145c960345f (127.18.39.65). New cstate: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } health_report { overall_health: UNKNOWN } } }
Jun 19 06:50:35 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851833, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:35.204972 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.191894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40955 (local address 127.0.0.1:47014)
0619 06:50:35.192521 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.192543 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.192712 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.193076 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.193081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.193856 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:35.194113 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.194122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.195171 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.195176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.196110 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.196121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.196871 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.196878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.197053 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.197762 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:35.197782 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:35.200023 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:35.202854 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:35.202865 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:35.202871 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:35.203261 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:35.203596 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:35.203599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:35.203601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:35.203722 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:35.204170 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:35.204176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:35.204342 (+ 166us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.204466 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.204663 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":329,"threads_started":1}
I20260619 06:50:35.205106 20424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.191908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.18.39.126:40955)
0619 06:50:35.192072 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.192097 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.192635 (+ 538us) server_negotiation.cc:407] Connection header received
0619 06:50:35.192840 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.192845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.192919 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.193049 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:35.194304 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.195025 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.196265 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.196684 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.196828 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.200208 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:35.200213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:35.200216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:35.200244 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:35.202718 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:35.203383 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:35.203387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:35.203389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:35.203505 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:35.203806 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:35.203810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:35.203812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:35.204068 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:35.204170 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.204553 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.204776 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":54}
I20260619 06:50:35.209772 20329 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:47014:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:50:35.210768 20329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:35.217381 20605 tablet_service.cc:1467] Processing CreateTablet for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5720433a3e154937a07c1d5ae08ed365]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:35.217273 20466 tablet_service.cc:1467] Processing CreateTablet for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5720433a3e154937a07c1d5ae08ed365]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:35.217674 20744 tablet_service.cc:1467] Processing CreateTablet for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5720433a3e154937a07c1d5ae08ed365]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:50:35.218333 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.218333 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.218447 20744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a38e1e62254e4ffeaa9f4a0061bcf4d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.224357 20833 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Bootstrap starting.
I20260619 06:50:35.225458 20836 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: Bootstrap starting.
I20260619 06:50:35.226141 20833 tablet_bootstrap.cc:654] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.226519 20835 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: Bootstrap starting.
I20260619 06:50:35.227057 20836 tablet_bootstrap.cc:654] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.228313 20835 tablet_bootstrap.cc:654] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.229084 20833 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: No bootstrap required, opened a new log
I20260619 06:50:35.229241 20833 ts_tablet_manager.cc:1397] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260619 06:50:35.230156 20836 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: No bootstrap required, opened a new log
I20260619 06:50:35.230135 20833 raft_consensus.cc:348] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.230336 20836 ts_tablet_manager.cc:1397] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 06:50:35.230406 20833 raft_consensus.cc:374] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.230499 20833 raft_consensus.cc:729] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.230765 20833 consensus_queue.cc:260] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.231395 20833 ts_tablet_manager.cc:1428] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 06:50:35.231244 20839 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.231335 20836 raft_consensus.cc:348] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.231626 20839 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.231649 20836 raft_consensus.cc:374] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.231755 20836 raft_consensus.cc:729] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab03559e33d1412ab413c145c960345f, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.232024 20836 consensus_queue.cc:260] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.232496 20841 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.232795 20841 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.232666 20836 ts_tablet_manager.cc:1428] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 06:50:35.234756 20835 tablet_bootstrap.cc:492] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: No bootstrap required, opened a new log
I20260619 06:50:35.235003 20835 ts_tablet_manager.cc:1397] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:50:35.235314 20841 raft_consensus.cc:482] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:35.235479 20841 raft_consensus.cc:504] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.235872 20835 raft_consensus.cc:348] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.236105 20835 raft_consensus.cc:374] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.236173 20841 leader_election.cc:288] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653), 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759)
I20260619 06:50:35.236202 20835 raft_consensus.cc:729] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.236425 20835 consensus_queue.cc:260] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.236691 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a38e1e62254e4ffeaa9f4a0061bcf4d3" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f" is_pre_election: true
I20260619 06:50:35.236940 20628 raft_consensus.cc:2454] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 0.
I20260619 06:50:35.236876 20840 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.237234 20840 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.237414 20417 leader_election.cc:302] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab03559e33d1412ab413c145c960345f, dcfa021d06264f368f49a9ee88eb825f; no voters:
I20260619 06:50:35.237529 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a38e1e62254e4ffeaa9f4a0061bcf4d3" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" is_pre_election: true
I20260619 06:50:35.237677 20841 raft_consensus.cc:2781] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:35.237787 20841 raft_consensus.cc:482] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:35.237775 20768 raft_consensus.cc:2454] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 0.
I20260619 06:50:35.237905 20841 raft_consensus.cc:3037] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.238320 20835 ts_tablet_manager.cc:1428] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:35.240733 20841 raft_consensus.cc:504] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.241417 20841 leader_election.cc:288] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 election: Requested vote from peers dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653), 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759)
I20260619 06:50:35.241793 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a38e1e62254e4ffeaa9f4a0061bcf4d3" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f"
I20260619 06:50:35.241870 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a38e1e62254e4ffeaa9f4a0061bcf4d3" candidate_uuid: "ab03559e33d1412ab413c145c960345f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
I20260619 06:50:35.242013 20628 raft_consensus.cc:3037] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.242046 20768 raft_consensus.cc:3037] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.243930 20628 raft_consensus.cc:2454] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 1.
I20260619 06:50:35.243937 20768 raft_consensus.cc:2454] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab03559e33d1412ab413c145c960345f in term 1.
I20260619 06:50:35.244302 20417 leader_election.cc:302] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab03559e33d1412ab413c145c960345f, dcfa021d06264f368f49a9ee88eb825f; no voters:
I20260619 06:50:35.244567 20841 raft_consensus.cc:2781] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:35.244769 20841 raft_consensus.cc:686] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [term 1 LEADER]: Becoming Leader. State: Replica: ab03559e33d1412ab413c145c960345f, State: Running, Role: LEADER
I20260619 06:50:35.245020 20841 consensus_queue.cc:237] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.246850 20844 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: TxnStatusTablet state changed. Reason: New leader ab03559e33d1412ab413c145c960345f. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.247262 20844 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:35.247996 20856 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:50:35.248267 20856 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:50:35.248629 20328 catalog_manager.cc:5495] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f reported cstate change: term changed from 0 to 1, leader changed from <none> to ab03559e33d1412ab413c145c960345f (127.18.39.65). New cstate: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } health_report { overall_health: UNKNOWN } } }
I20260619 06:50:35.266748 20851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.260660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.0.0.1:48608)
0619 06:50:35.260993 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.261008 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.261176 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.261686 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.261692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.261704 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:35.261888 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.261896 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.263032 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.263039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.264355 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.264367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.265058 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.265064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.265245 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.265305 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.266189 (+ 884us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.266210 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.266294 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.266442 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260619 06:50:35.266719 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.260652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.18.39.65:41235)
0619 06:50:35.260838 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.260844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.261086 (+ 242us) server_negotiation.cc:407] Connection header received
0619 06:50:35.261405 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.261410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.261522 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.261675 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:35.262023 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.262892 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.264487 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.264899 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.265047 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.265547 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.266067 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.266225 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.266311 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.266391 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:50:35.275015 20768 raft_consensus.cc:1261] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Refusing update from remote peer ab03559e33d1412ab413c145c960345f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.275081 20628 raft_consensus.cc:1261] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Refusing update from remote peer ab03559e33d1412ab413c145c960345f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.276050 20844 consensus_queue.cc:1035] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:35.276422 20841 consensus_queue.cc:1035] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:35.284807 20840 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: TxnStatusTablet state changed. Reason: New leader ab03559e33d1412ab413c145c960345f. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.285138 20840 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.288975 20859 mvcc.cc:205] Tried to move back new op lower bound from 7298465117275652096 to 7298465117165760512. Current Snapshot: MvccSnapshot[applied={T|T < 7298465117275652096}]
I20260619 06:50:35.290861 20840 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.291229 20840 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P dcfa021d06264f368f49a9ee88eb825f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.296080 20839 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: TxnStatusTablet state changed. Reason: New leader ab03559e33d1412ab413c145c960345f. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.296358 20839 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.298709 20844 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.299013 20844 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:35.303140 20844 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.303514 20844 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P ab03559e33d1412ab413c145c960345f: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:50:35.308146 20839 tablet_replica.cc:439] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ab03559e33d1412ab413c145c960345f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } } }
I20260619 06:50:35.308410 20839 tablet_replica.cc:442] T a38e1e62254e4ffeaa9f4a0061bcf4d3 P 9a5d41b4f8b04d7cb80832c9b05c0f18: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:50:35.316144 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.311365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.18.39.65:41235)
0619 06:50:35.311540 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.311546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.311561 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:50:35.311626 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.311629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.311687 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.311820 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:35.312192 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.312971 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.314380 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.314821 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.314986 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.315413 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.315528 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.315630 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.315778 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.315858 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:50:35.334385 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.327832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:33493 (local address 127.18.39.65:41235)
0619 06:50:35.328020 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.328026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.328308 (+ 282us) server_negotiation.cc:407] Connection header received
0619 06:50:35.328518 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.328522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.328579 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.328652 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:35.329378 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.330161 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.332506 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.332940 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.333035 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.333668 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.333784 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.333849 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.334081 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.334144 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:50:35.334738 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.327421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.18.39.65:33493)
0619 06:50:35.328197 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.328219 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.328398 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.328699 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.328705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.328719 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:35.329153 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.329166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.330297 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.330303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.332363 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.332378 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.333134 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.333141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.333396 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.333469 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.333886 (+ 417us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.333898 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.334003 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.334151 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":389,"threads_started":1}
I20260619 06:50:35.344209 20627 raft_consensus.cc:1261] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Refusing update from remote peer ab03559e33d1412ab413c145c960345f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.344861 20768 raft_consensus.cc:1261] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Refusing update from remote peer ab03559e33d1412ab413c145c960345f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.345516 20871 consensus_queue.cc:1035] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:35.345921 20841 consensus_queue.cc:1035] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:35.353663 20861 mvcc.cc:205] Tried to move back new op lower bound from 7298465117564669952 to 7298465116600369152. Current Snapshot: MvccSnapshot[applied={T|T < 7298465117564669952}]
I20260619 06:50:35.362825 20860 mvcc.cc:205] Tried to move back new op lower bound from 7298465117564669952 to 7298465116600369152. Current Snapshot: MvccSnapshot[applied={T|T < 7298465117564669952}]
I20260619 06:50:35.419952 20329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:35.420231 20329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:35.424252 20329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Sending DeleteTablet for 3 replicas of tablet 697dabe7791f4075b71e772f8ca511c1
I20260619 06:50:35.425462 20744 tablet_service.cc:1514] Processing DeleteTablet for tablet 697dabe7791f4075b71e772f8ca511c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:35 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 06:50:35.425580 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet 697dabe7791f4075b71e772f8ca511c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:35 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60430
I20260619 06:50:35.425801 20465 tablet_service.cc:1514] Processing DeleteTablet for tablet 697dabe7791f4075b71e772f8ca511c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:35 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:48598
I20260619 06:50:35.427865 20329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:35.428821 20329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:35.437211 20877 tablet_replica.cc:330] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: stopping tablet replica
I20260619 06:50:35.437842 20877 raft_consensus.cc:2229] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:35.438254 20877 raft_consensus.cc:2258] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:35.438417 20876 tablet_replica.cc:330] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: stopping tablet replica
I20260619 06:50:35.438912 20876 raft_consensus.cc:2229] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:35.439428 20744 tablet_service.cc:1467] Processing CreateTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 (DEFAULT_TABLE table=test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:35.439908 20605 tablet_service.cc:1467] Processing CreateTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 (DEFAULT_TABLE table=test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:35.439985 20466 tablet_service.cc:1467] Processing CreateTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 (DEFAULT_TABLE table=test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:35.440033 20744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa4b2ce78ad4971af21b45c9b171635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.440433 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa4b2ce78ad4971af21b45c9b171635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.440474 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa4b2ce78ad4971af21b45c9b171635. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:35.444388 20876 raft_consensus.cc:2258] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:35.449998 20836 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Bootstrap starting.
I20260619 06:50:35.450414 20876 ts_tablet_manager.cc:1905] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:35.450639 20835 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Bootstrap starting.
I20260619 06:50:35.451509 20877 ts_tablet_manager.cc:1905] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:35.452042 20836 tablet_bootstrap.cc:654] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.452345 20835 tablet_bootstrap.cc:654] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.452389 20878 tablet_replica.cc:330] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: stopping tablet replica
I20260619 06:50:35.452975 20878 raft_consensus.cc:2229] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:35.461285 20877 ts_tablet_manager.cc:1918] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:35.461562 20877 log.cc:1196] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/697dabe7791f4075b71e772f8ca511c1
I20260619 06:50:35.462284 20877 ts_tablet_manager.cc:1939] T 697dabe7791f4075b71e772f8ca511c1 P dcfa021d06264f368f49a9ee88eb825f: Deleting consensus metadata
I20260619 06:50:35.464627 20314 catalog_manager.cc:4841] TS dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653): tablet 697dabe7791f4075b71e772f8ca511c1 (table test-table [id=0c38ef09229b42488f9f4c94d147ee4d]) successfully deleted
I20260619 06:50:35.465128 20833 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Bootstrap starting.
I20260619 06:50:35.465305 20878 raft_consensus.cc:2258] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:35.466907 20833 tablet_bootstrap.cc:654] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:35.467715 20878 ts_tablet_manager.cc:1905] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:35.468081 20835 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: No bootstrap required, opened a new log
I20260619 06:50:35.468281 20835 ts_tablet_manager.cc:1397] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.006s
I20260619 06:50:35.469128 20835 raft_consensus.cc:348] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.469455 20835 raft_consensus.cc:374] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.469543 20835 raft_consensus.cc:729] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.469888 20835 consensus_queue.cc:260] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.471407 20835 ts_tablet_manager.cc:1428] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:35.471825 20876 ts_tablet_manager.cc:1918] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:35.472044 20876 log.cc:1196] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/697dabe7791f4075b71e772f8ca511c1
I20260619 06:50:35.472741 20876 ts_tablet_manager.cc:1939] T 697dabe7791f4075b71e772f8ca511c1 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting consensus metadata
I20260619 06:50:35.473799 20836 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: No bootstrap required, opened a new log
I20260619 06:50:35.473981 20836 ts_tablet_manager.cc:1397] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.008s
I20260619 06:50:35.474808 20836 raft_consensus.cc:348] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.475088 20836 raft_consensus.cc:374] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.475183 20836 raft_consensus.cc:729] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab03559e33d1412ab413c145c960345f, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.475479 20836 consensus_queue.cc:260] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.477008 20836 ts_tablet_manager.cc:1428] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:35.478533 20314 catalog_manager.cc:4841] TS 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759): tablet 697dabe7791f4075b71e772f8ca511c1 (table test-table [id=0c38ef09229b42488f9f4c94d147ee4d]) successfully deleted
I20260619 06:50:35.479521 20833 tablet_bootstrap.cc:492] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: No bootstrap required, opened a new log
I20260619 06:50:35.479703 20833 ts_tablet_manager.cc:1397] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.007s
I20260619 06:50:35.480511 20833 raft_consensus.cc:348] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.480792 20833 raft_consensus.cc:374] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:35.480888 20833 raft_consensus.cc:729] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Initialized, Role: FOLLOWER
I20260619 06:50:35.481154 20833 consensus_queue.cc:260] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.481825 20833 ts_tablet_manager.cc:1428] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 06:50:35.485181 20878 ts_tablet_manager.cc:1918] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:35.485436 20878 log.cc:1196] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/697dabe7791f4075b71e772f8ca511c1
I20260619 06:50:35.485993 20878 ts_tablet_manager.cc:1939] T 697dabe7791f4075b71e772f8ca511c1 P ab03559e33d1412ab413c145c960345f: Deleting consensus metadata
I20260619 06:50:35.487598 20314 catalog_manager.cc:4841] TS ab03559e33d1412ab413c145c960345f (127.18.39.65:41235): tablet 697dabe7791f4075b71e772f8ca511c1 (table test-table [id=0c38ef09229b42488f9f4c94d147ee4d]) successfully deleted
I20260619 06:50:35.777053 20840 raft_consensus.cc:482] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:35.777313 20840 raft_consensus.cc:504] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.779219 20840 leader_election.cc:288] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759)
I20260619 06:50:35.787299 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.780336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41759 (local address 127.18.39.66:60291)
0619 06:50:35.780864 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.780882 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.781036 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.781885 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.781890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.781919 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.782271 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.782283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.783369 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.783377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.784773 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.784785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.786422 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.786429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.786629 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.786672 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.786780 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.786921 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":4,"thread_start_us":396,"threads_started":1}
I20260619 06:50:35.787397 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.780612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:60291 (local address 127.18.39.67:41759)
0619 06:50:35.781553 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.781559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.781583 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:50:35.781656 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.781664 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.781738 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.781841 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.782467 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.783226 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.784941 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.786231 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.786412 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.786492 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.786860 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.787072 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":715,"threads_started":1}
I20260619 06:50:35.787637 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.779707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:39751 (local address 127.18.39.65:41235)
0619 06:50:35.779949 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.779956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.780253 (+ 297us) server_negotiation.cc:407] Connection header received
0619 06:50:35.780593 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.780599 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.780682 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.780789 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.781557 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.782518 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.785090 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.786489 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.786673 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.786756 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.787228 (+ 472us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.787376 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 06:50:35.788389 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.779482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.18.39.66:39751)
0619 06:50:35.780131 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.780149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.780344 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.780984 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.780990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.781023 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.781297 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.781308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.782669 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.782674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.784858 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.784873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.786733 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.786739 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.786931 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.786970 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.787678 (+ 708us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.787843 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":47,"thread_start_us":429,"threads_started":1}
I20260619 06:50:35.788446 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "dcfa021d06264f368f49a9ee88eb825f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" is_pre_election: true
I20260619 06:50:35.788704 20768 raft_consensus.cc:2454] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfa021d06264f368f49a9ee88eb825f in term 0.
I20260619 06:50:35.789242 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "dcfa021d06264f368f49a9ee88eb825f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f" is_pre_election: true
I20260619 06:50:35.789542 20557 leader_election.cc:302] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, dcfa021d06264f368f49a9ee88eb825f; no voters:
I20260619 06:50:35.789669 20486 raft_consensus.cc:2454] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcfa021d06264f368f49a9ee88eb825f in term 0.
I20260619 06:50:35.789971 20840 raft_consensus.cc:2781] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:35.790071 20840 raft_consensus.cc:482] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:35.790160 20840 raft_consensus.cc:3037] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.791730 20839 raft_consensus.cc:482] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:35.791936 20839 raft_consensus.cc:504] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.793311 20840 raft_consensus.cc:504] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.793728 20839 leader_election.cc:288] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:35.793942 20840 leader_election.cc:288] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [CANDIDATE]: Term 1 election: Requested vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759)
I20260619 06:50:35.794963 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "dcfa021d06264f368f49a9ee88eb825f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f"
I20260619 06:50:35.795176 20486 raft_consensus.cc:3037] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.797773 20768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "dcfa021d06264f368f49a9ee88eb825f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18"
I20260619 06:50:35.798002 20768 raft_consensus.cc:3037] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:35.802925 20486 raft_consensus.cc:2454] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfa021d06264f368f49a9ee88eb825f in term 1.
I20260619 06:50:35.803483 20557 leader_election.cc:302] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab03559e33d1412ab413c145c960345f, dcfa021d06264f368f49a9ee88eb825f; no voters:
I20260619 06:50:35.803838 20840 raft_consensus.cc:2781] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:35.801281 20768 raft_consensus.cc:2454] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcfa021d06264f368f49a9ee88eb825f in term 1.
I20260619 06:50:35.815768 20840 raft_consensus.cc:686] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 1 LEADER]: Becoming Leader. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Running, Role: LEADER
I20260619 06:50:35.816535 20840 consensus_queue.cc:237] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:35.819262 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.794597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:52335 (local address 127.18.39.65:41235)
0619 06:50:35.794768 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.794774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.801889 (+ 7115us) server_negotiation.cc:407] Connection header received
0619 06:50:35.802097 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.802102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.802171 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.802249 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.807034 (+ 4785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.808135 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.814078 (+ 5943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.815401 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.815573 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.815650 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.818777 (+ 3127us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.818954 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:50:35.820039 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.794561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.18.39.67:52335)
0619 06:50:35.801772 (+ 7211us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.801796 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.801974 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.805815 (+ 3841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.805822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.805859 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.806214 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.806225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.808351 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.808357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.812334 (+ 3977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.812354 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.818335 (+ 5981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.818342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.818571 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.818608 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.819526 (+ 918us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.819694 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7049,"thread_start_us":6918,"threads_started":1}
I20260619 06:50:35.824999 20329 catalog_manager.cc:5495] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f reported cstate change: term changed from 0 to 1, leader changed from <none> to dcfa021d06264f368f49a9ee88eb825f (127.18.39.66). New cstate: current_term: 1 leader_uuid: "dcfa021d06264f368f49a9ee88eb825f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } health_report { overall_health: UNKNOWN } } }
I20260619 06:50:35.830293 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f" is_pre_election: true
I20260619 06:50:35.830744 20486 raft_consensus.cc:2379] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in current term 1: Already voted for candidate dcfa021d06264f368f49a9ee88eb825f in this term.
I20260619 06:50:35.835881 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.802101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:43699 (local address 127.18.39.66:39653)
0619 06:50:35.823664 (+ 21563us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.823670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.823693 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:50:35.823755 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.823760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.823827 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.823920 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:35.830370 (+ 6450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.831648 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.833970 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.834852 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.835007 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.835060 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.835512 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.835617 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21426,"thread_start_us":21302,"threads_started":1}
I20260619 06:50:35.835918 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.802109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39653 (local address 127.18.39.67:43699)
0619 06:50:35.820271 (+ 18162us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.820291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.820447 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.824174 (+ 3727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.824179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.824208 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:35.830074 (+ 5866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.830090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.831812 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.831818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.833770 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.833789 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.835026 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.835032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.835255 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.835294 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.835453 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.835607 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18060,"thread_start_us":27294,"threads_started":1}
I20260619 06:50:35.836578 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa4b2ce78ad4971af21b45c9b171635" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f" is_pre_election: true
I20260619 06:50:35.837208 20696 leader_election.cc:302] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18; no voters: ab03559e33d1412ab413c145c960345f, dcfa021d06264f368f49a9ee88eb825f
I20260619 06:50:35.837713 20839 raft_consensus.cc:2726] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 06:50:35.939064 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.933312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.18.39.66:39653)
0619 06:50:35.933553 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.933558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.933996 (+ 438us) server_negotiation.cc:407] Connection header received
0619 06:50:35.934237 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.934242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.934323 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.934481 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:35.934951 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.935650 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.936933 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.937365 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.937504 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.937927 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.938473 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.938652 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.938716 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.938795 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:50:35.955303 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.949654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:53603 (local address 127.18.39.65:41235)
0619 06:50:35.949825 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.949849 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:35.950352 (+ 503us) server_negotiation.cc:407] Connection header received
0619 06:50:35.950561 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:35.950568 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:35.950642 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:35.950736 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:35.951269 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.952048 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.953451 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.953883 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.954022 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.954542 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.954659 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.954763 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:35.954909 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 06:50:35.954983 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:50:35.955403 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.949647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.18.39.66:53603)
0619 06:50:35.950239 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:35.950254 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:35.950426 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:35.950762 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:35.950767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:35.950779 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:35.951097 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.951106 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.952191 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.952197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.953261 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:35.953272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:35.954076 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:35.954083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:35.954287 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:35.954354 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:35.954779 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:35.954791 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:50:35.954892 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:50:35.955026 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":318,"threads_started":1}
I20260619 06:50:35.964321 20768 raft_consensus.cc:1261] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Refusing update from remote peer dcfa021d06264f368f49a9ee88eb825f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.964401 20486 raft_consensus.cc:1261] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Refusing update from remote peer dcfa021d06264f368f49a9ee88eb825f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:35.965035 20888 consensus_queue.cc:1035] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:35.965440 20840 consensus_queue.cc:1035] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:36.012569 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.998335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:34663 (local address 127.18.39.66:39653)
0619 06:50:35.998542 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:35.998551 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:36.005973 (+ 7422us) server_negotiation.cc:407] Connection header received
0619 06:50:36.006283 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:36.006289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:36.006377 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:36.006496 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:36.007412 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:36.008198 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:36.010228 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:36.010642 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:36.010780 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:36.011729 (+ 949us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:36.011914 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:36.012012 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:36.012170 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 06:50:36.012247 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:50:36.012910 20907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:35.998246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:39653 (local address 127.18.39.65:34663)
0619 06:50:36.005856 (+ 7610us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:36.005885 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:36.006091 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:36.006558 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:36.006563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:36.006576 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:36.007246 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:36.007258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:36.008351 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:36.008357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:36.010076 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:36.010091 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:36.011277 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:36.011283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:36.011476 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:36.011536 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:36.012033 (+ 497us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:36.012045 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:50:36.012146 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:50:36.012281 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7439,"thread_start_us":7290,"threads_started":1}
W20260619 06:50:36.014209 20893 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 06:50:37.634758 20327 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:37.635075 20327 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47008:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:37.639533 20327 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Sending DeleteTablet for 3 replicas of tablet cfa4b2ce78ad4971af21b45c9b171635
I20260619 06:50:37.640508 20744 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 06:50:37.640494 20466 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:48598
I20260619 06:50:37.640522 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa4b2ce78ad4971af21b45c9b171635 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60430
I20260619 06:50:37.651264 20934 tablet_replica.cc:330] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: stopping tablet replica
I20260619 06:50:37.651510 20934 raft_consensus.cc:2229] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:37.651988 20934 raft_consensus.cc:2258] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.652618 20932 tablet_replica.cc:330] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: stopping tablet replica
I20260619 06:50:37.653080 20933 tablet_replica.cc:330] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: stopping tablet replica
I20260619 06:50:37.653208 20934 ts_tablet_manager.cc:1905] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:37.653294 20933 raft_consensus.cc:2229] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:37.653458 20932 raft_consensus.cc:2229] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:37.653607 20933 raft_consensus.cc:2258] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.653813 20932 raft_consensus.cc:2258] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.654556 20933 ts_tablet_manager.cc:1905] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:37.655193 20932 ts_tablet_manager.cc:1905] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:37.660043 20934 ts_tablet_manager.cc:1918] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:50:37.660669 20934 log.cc:1196] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/cfa4b2ce78ad4971af21b45c9b171635
I20260619 06:50:37.661193 20934 ts_tablet_manager.cc:1939] T cfa4b2ce78ad4971af21b45c9b171635 P dcfa021d06264f368f49a9ee88eb825f: Deleting consensus metadata
I20260619 06:50:37.661803 20933 ts_tablet_manager.cc:1918] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:50:37.661965 20933 log.cc:1196] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/cfa4b2ce78ad4971af21b45c9b171635
I20260619 06:50:37.662570 20933 ts_tablet_manager.cc:1939] T cfa4b2ce78ad4971af21b45c9b171635 P ab03559e33d1412ab413c145c960345f: Deleting consensus metadata
I20260619 06:50:37.662628 20314 catalog_manager.cc:4841] TS dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653): tablet cfa4b2ce78ad4971af21b45c9b171635 (table test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]) successfully deleted
I20260619 06:50:37.663203 20932 ts_tablet_manager.cc:1918] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:50:37.663335 20932 log.cc:1196] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/cfa4b2ce78ad4971af21b45c9b171635
I20260619 06:50:37.663659 20314 catalog_manager.cc:4841] TS ab03559e33d1412ab413c145c960345f (127.18.39.65:41235): tablet cfa4b2ce78ad4971af21b45c9b171635 (table test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]) successfully deleted
I20260619 06:50:37.663767 20932 ts_tablet_manager.cc:1939] T cfa4b2ce78ad4971af21b45c9b171635 P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting consensus metadata
I20260619 06:50:37.664660 20314 catalog_manager.cc:4841] TS 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759): tablet cfa4b2ce78ad4971af21b45c9b171635 (table test-table [id=91d6524c5ac2472c9aa36712c4d4cdb8]) successfully deleted
Jun 19 06:50:37 dist-test-slave-1h5s krb5kdc[20284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:50:37 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:37.698539 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.683887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.18.39.126:40955)
0619 06:50:37.684443 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:37.684451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:37.684577 (+ 126us) server_negotiation.cc:407] Connection header received
0619 06:50:37.684830 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:37.684838 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:37.684943 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:37.685070 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:37.686587 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.687374 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.688620 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.689051 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.689193 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.693239 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:37.693244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:37.693248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:37.693283 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:37.695792 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:37.696486 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:37.696493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:37.696497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:37.696619 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:37.696985 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:37.696992 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:37.696995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:37.697272 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:37.697537 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:37.697897 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:50:37.698114 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":264,"threads_started":1}
I20260619 06:50:37.701917 20327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:37.702463 20327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:37.708781 20466 tablet_service.cc:1467] Processing CreateTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb (DEFAULT_TABLE table=test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:37.708969 20744 tablet_service.cc:1467] Processing CreateTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb (DEFAULT_TABLE table=test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:37.709158 20605 tablet_service.cc:1467] Processing CreateTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb (DEFAULT_TABLE table=test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:37.709282 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0cd9e6974cf4f64bad1b83cd3e598fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:37.709378 20744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0cd9e6974cf4f64bad1b83cd3e598fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:37.709578 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0cd9e6974cf4f64bad1b83cd3e598fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:37.720152 20944 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Bootstrap starting.
I20260619 06:50:37.725045 20944 tablet_bootstrap.cc:654] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:37.725679 20943 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Bootstrap starting.
I20260619 06:50:37.727942 20943 tablet_bootstrap.cc:654] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:37.734944 20943 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: No bootstrap required, opened a new log
I20260619 06:50:37.735133 20943 ts_tablet_manager.cc:1397] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260619 06:50:37.735687 20944 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: No bootstrap required, opened a new log
I20260619 06:50:37.735858 20944 ts_tablet_manager.cc:1397] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.006s
I20260619 06:50:37.735854 20943 raft_consensus.cc:348] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.736028 20945 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Bootstrap starting.
I20260619 06:50:37.736095 20943 raft_consensus.cc:374] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:37.736194 20943 raft_consensus.cc:729] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Initialized, Role: FOLLOWER
I20260619 06:50:37.736464 20943 consensus_queue.cc:260] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.736631 20944 raft_consensus.cc:348] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.736903 20944 raft_consensus.cc:374] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:37.737011 20944 raft_consensus.cc:729] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab03559e33d1412ab413c145c960345f, State: Initialized, Role: FOLLOWER
I20260619 06:50:37.737262 20944 consensus_queue.cc:260] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.738523 20945 tablet_bootstrap.cc:654] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:37.739166 20944 ts_tablet_manager.cc:1428] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:37.741007 20943 ts_tablet_manager.cc:1428] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260619 06:50:37.744287 20945 tablet_bootstrap.cc:492] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: No bootstrap required, opened a new log
I20260619 06:50:37.744436 20945 ts_tablet_manager.cc:1397] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:50:37.745151 20945 raft_consensus.cc:348] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.745422 20945 raft_consensus.cc:374] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:37.745741 20945 raft_consensus.cc:729] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Initialized, Role: FOLLOWER
I20260619 06:50:37.745994 20945 consensus_queue.cc:260] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.746634 20945 ts_tablet_manager.cc:1428] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:50:37.833963 20948 raft_consensus.cc:482] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:37.834224 20948 raft_consensus.cc:504] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.834949 20948 leader_election.cc:288] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:37.835546 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0cd9e6974cf4f64bad1b83cd3e598fb" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f" is_pre_election: true
I20260619 06:50:37.835624 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0cd9e6974cf4f64bad1b83cd3e598fb" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f" is_pre_election: true
I20260619 06:50:37.835786 20486 raft_consensus.cc:2454] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 0.
I20260619 06:50:37.835875 20628 raft_consensus.cc:2454] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 0.
I20260619 06:50:37.836259 20696 leader_election.cc:302] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, ab03559e33d1412ab413c145c960345f; no voters:
I20260619 06:50:37.836530 20948 raft_consensus.cc:2781] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:37.836658 20948 raft_consensus.cc:482] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:37.836771 20948 raft_consensus.cc:3037] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:37.839005 20948 raft_consensus.cc:504] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.839656 20948 leader_election.cc:288] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 election: Requested vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:37.840183 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0cd9e6974cf4f64bad1b83cd3e598fb" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f"
I20260619 06:50:37.840147 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0cd9e6974cf4f64bad1b83cd3e598fb" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f"
I20260619 06:50:37.840404 20628 raft_consensus.cc:3037] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:37.840415 20486 raft_consensus.cc:3037] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:37.842507 20628 raft_consensus.cc:2454] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 1.
I20260619 06:50:37.842669 20486 raft_consensus.cc:2454] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 1.
I20260619 06:50:37.842974 20696 leader_election.cc:302] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, dcfa021d06264f368f49a9ee88eb825f; no voters:
I20260619 06:50:37.843284 20948 raft_consensus.cc:2781] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:37.843711 20948 raft_consensus.cc:686] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Running, Role: LEADER
I20260619 06:50:37.844177 20948 consensus_queue.cc:237] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:37.847877 20329 catalog_manager.cc:5495] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } health_report { overall_health: HEALTHY } } }
I20260619 06:50:37.896240 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.890500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.18.39.67:41759)
0619 06:50:37.891099 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:37.891103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:37.891123 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:50:37.891227 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:37.891232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:37.891307 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:37.891461 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:37.891923 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.892787 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.894082 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.894472 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.894562 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.894941 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:37.895548 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:37.895664 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:37.895773 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 06:50:37.895838 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":372,"threads_started":1}
I20260619 06:50:37.911773 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.906324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:59815 (local address 127.18.39.65:41235)
0619 06:50:37.907301 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:37.907305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:37.907324 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:50:37.907382 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:37.907386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:37.907454 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:37.907535 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:37.908032 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.908793 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.910225 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.910649 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.910743 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.911163 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:37.911269 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:37.911337 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:37.911461 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 06:50:37.911520 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":335,"threads_started":1}
I20260619 06:50:37.911794 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.906119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41235 (local address 127.18.39.67:59815)
0619 06:50:37.906790 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:37.906807 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:37.906916 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:37.907586 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:37.907589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:37.907600 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:37.907867 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:37.907874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.908932 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.908936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:37.910107 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:37.910119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.910815 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.910820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:37.910930 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.910985 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:37.911355 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:37.911363 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:50:37.911428 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 06:50:37.911522 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":333,"threads_started":1}
I20260619 06:50:37.920259 20486 raft_consensus.cc:1261] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Refusing update from remote peer 9a5d41b4f8b04d7cb80832c9b05c0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:37.920289 20628 raft_consensus.cc:1261] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Refusing update from remote peer 9a5d41b4f8b04d7cb80832c9b05c0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:37.920889 20950 consensus_queue.cc:1035] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:37.921227 20948 consensus_queue.cc:1035] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:37.949994 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.944299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:35267 (local address 127.18.39.67:41759)
0619 06:50:37.944473 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:37.944477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:37.945004 (+ 527us) server_negotiation.cc:407] Connection header received
0619 06:50:37.945191 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:37.945199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:37.945318 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:37.945443 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:37.945928 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.946630 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.948338 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.948916 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.949013 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.949430 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:37.949528 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:37.949596 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:37.949723 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 06:50:37.949786 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:50:37.950133 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:37.944208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41759 (local address 127.18.39.65:35267)
0619 06:50:37.944908 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:37.944923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:37.945073 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:37.945460 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:37.945464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:37.945476 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:50:37.945758 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:37.945767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.946770 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.946777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:37.948152 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:37.948163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:37.949072 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:37.949077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:37.949196 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:37.949235 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:50:37.949614 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:50:37.949623 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 06:50:37.949689 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 06:50:37.949791 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":421,"threads_started":1}
I20260619 06:50:37.984625 20329 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:37.984905 20329 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:37.988073 20329 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Sending DeleteTablet for 3 replicas of tablet f0cd9e6974cf4f64bad1b83cd3e598fb
I20260619 06:50:37.988919 20466 tablet_service.cc:1514] Processing DeleteTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:48598
I20260619 06:50:37.989219 20933 tablet_replica.cc:330] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: stopping tablet replica
I20260619 06:50:37.989449 20933 raft_consensus.cc:2229] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:37.989527 20744 tablet_service.cc:1514] Processing DeleteTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 06:50:37.989650 20933 raft_consensus.cc:2258] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.989856 20932 tablet_replica.cc:330] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: stopping tablet replica
I20260619 06:50:37.990074 20932 raft_consensus.cc:2229] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:37.990593 20932 raft_consensus.cc:2258] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.990639 20329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 06:50:37.991042 20933 ts_tablet_manager.cc:1905] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 06:50:37.991339 20329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:37.991355 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet f0cd9e6974cf4f64bad1b83cd3e598fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:37 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60430
I20260619 06:50:37.991618 20934 tablet_replica.cc:330] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: stopping tablet replica
I20260619 06:50:37.991823 20934 raft_consensus.cc:2229] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:37.992005 20934 raft_consensus.cc:2258] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:37.992471 20932 ts_tablet_manager.cc:1905] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:37.993132 20934 ts_tablet_manager.cc:1905] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:37.998199 20933 ts_tablet_manager.cc:1918] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:37.998477 20933 log.cc:1196] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/f0cd9e6974cf4f64bad1b83cd3e598fb
I20260619 06:50:37.999012 20933 ts_tablet_manager.cc:1939] T f0cd9e6974cf4f64bad1b83cd3e598fb P ab03559e33d1412ab413c145c960345f: Deleting consensus metadata
I20260619 06:50:37.999550 20932 ts_tablet_manager.cc:1918] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:37.999718 20932 log.cc:1196] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/f0cd9e6974cf4f64bad1b83cd3e598fb
I20260619 06:50:38.000144 20932 ts_tablet_manager.cc:1939] T f0cd9e6974cf4f64bad1b83cd3e598fb P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting consensus metadata
I20260619 06:50:38.000737 20314 catalog_manager.cc:4841] TS ab03559e33d1412ab413c145c960345f (127.18.39.65:41235): tablet f0cd9e6974cf4f64bad1b83cd3e598fb (table test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]) successfully deleted
I20260619 06:50:38.000867 20934 ts_tablet_manager.cc:1918] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:50:38.001012 20934 log.cc:1196] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/f0cd9e6974cf4f64bad1b83cd3e598fb
I20260619 06:50:38.001209 20314 catalog_manager.cc:4841] TS 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759): tablet f0cd9e6974cf4f64bad1b83cd3e598fb (table test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]) successfully deleted
I20260619 06:50:38.002578 20934 ts_tablet_manager.cc:1939] T f0cd9e6974cf4f64bad1b83cd3e598fb P dcfa021d06264f368f49a9ee88eb825f: Deleting consensus metadata
I20260619 06:50:38.003610 20314 catalog_manager.cc:4841] TS dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653): tablet f0cd9e6974cf4f64bad1b83cd3e598fb (table test-table [id=d1bd7dab2c3e4fdf954b4175578fe518]) successfully deleted
I20260619 06:50:38.007879 20605 tablet_service.cc:1467] Processing CreateTablet for tablet 82721e93de4b49b9a65bc1394b4359ee (DEFAULT_TABLE table=test-table [id=25fa2f16be5548229f8dbcc87f161969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:38.008001 20744 tablet_service.cc:1467] Processing CreateTablet for tablet 82721e93de4b49b9a65bc1394b4359ee (DEFAULT_TABLE table=test-table [id=25fa2f16be5548229f8dbcc87f161969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:38.008293 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82721e93de4b49b9a65bc1394b4359ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:38.008446 20744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82721e93de4b49b9a65bc1394b4359ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:38.013042 20466 tablet_service.cc:1467] Processing CreateTablet for tablet 82721e93de4b49b9a65bc1394b4359ee (DEFAULT_TABLE table=test-table [id=25fa2f16be5548229f8dbcc87f161969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:38.013231 20945 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Bootstrap starting.
I20260619 06:50:38.013613 20466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82721e93de4b49b9a65bc1394b4359ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:38.015337 20945 tablet_bootstrap.cc:654] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:38.018725 20943 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Bootstrap starting.
I20260619 06:50:38.019743 20944 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Bootstrap starting.
I20260619 06:50:38.020467 20943 tablet_bootstrap.cc:654] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:38.021729 20944 tablet_bootstrap.cc:654] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:38.028656 20943 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: No bootstrap required, opened a new log
I20260619 06:50:38.028944 20943 ts_tablet_manager.cc:1397] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260619 06:50:38.029834 20943 raft_consensus.cc:348] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.030246 20943 raft_consensus.cc:374] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:38.030382 20943 raft_consensus.cc:729] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Initialized, Role: FOLLOWER
I20260619 06:50:38.030725 20943 consensus_queue.cc:260] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.031255 20945 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: No bootstrap required, opened a new log
I20260619 06:50:38.031469 20945 ts_tablet_manager.cc:1397] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.007s
I20260619 06:50:38.032399 20945 raft_consensus.cc:348] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.032673 20945 raft_consensus.cc:374] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:38.032778 20945 raft_consensus.cc:729] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcfa021d06264f368f49a9ee88eb825f, State: Initialized, Role: FOLLOWER
I20260619 06:50:38.033030 20945 consensus_queue.cc:260] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.033516 20944 tablet_bootstrap.cc:492] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: No bootstrap required, opened a new log
I20260619 06:50:38.033627 20945 ts_tablet_manager.cc:1428] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 06:50:38.033686 20944 ts_tablet_manager.cc:1397] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260619 06:50:38.034399 20944 raft_consensus.cc:348] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.034626 20944 raft_consensus.cc:374] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:38.034727 20944 raft_consensus.cc:729] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab03559e33d1412ab413c145c960345f, State: Initialized, Role: FOLLOWER
I20260619 06:50:38.035028 20944 consensus_queue.cc:260] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.035341 20943 ts_tablet_manager.cc:1428] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260619 06:50:38.036259 20944 ts_tablet_manager.cc:1428] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:50:38.076845 20950 raft_consensus.cc:482] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:38.077136 20950 raft_consensus.cc:504] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.077960 20950 leader_election.cc:288] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:38.078531 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82721e93de4b49b9a65bc1394b4359ee" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f" is_pre_election: true
I20260619 06:50:38.078569 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82721e93de4b49b9a65bc1394b4359ee" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f" is_pre_election: true
I20260619 06:50:38.078802 20486 raft_consensus.cc:2454] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 0.
I20260619 06:50:38.078819 20628 raft_consensus.cc:2454] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 0.
I20260619 06:50:38.079250 20696 leader_election.cc:302] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, ab03559e33d1412ab413c145c960345f; no voters:
I20260619 06:50:38.079540 20950 raft_consensus.cc:2781] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:38.079645 20950 raft_consensus.cc:482] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:38.079774 20950 raft_consensus.cc:3037] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:38.082762 20950 raft_consensus.cc:504] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.083415 20950 leader_election.cc:288] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 election: Requested vote from peers ab03559e33d1412ab413c145c960345f (127.18.39.65:41235), dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653)
I20260619 06:50:38.083920 20486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82721e93de4b49b9a65bc1394b4359ee" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab03559e33d1412ab413c145c960345f"
I20260619 06:50:38.084043 20628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82721e93de4b49b9a65bc1394b4359ee" candidate_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcfa021d06264f368f49a9ee88eb825f"
I20260619 06:50:38.084142 20486 raft_consensus.cc:3037] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:38.084264 20628 raft_consensus.cc:3037] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:38.086248 20486 raft_consensus.cc:2454] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 1.
I20260619 06:50:38.086697 20696 leader_election.cc:302] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a5d41b4f8b04d7cb80832c9b05c0f18, ab03559e33d1412ab413c145c960345f; no voters:
I20260619 06:50:38.086979 20950 raft_consensus.cc:2781] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:38.087229 20628 raft_consensus.cc:2454] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a5d41b4f8b04d7cb80832c9b05c0f18 in term 1.
I20260619 06:50:38.087186 20950 raft_consensus.cc:686] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 LEADER]: Becoming Leader. State: Replica: 9a5d41b4f8b04d7cb80832c9b05c0f18, State: Running, Role: LEADER
I20260619 06:50:38.087764 20950 consensus_queue.cc:237] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } }
I20260619 06:50:38.090699 20327 catalog_manager.cc:5495] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a5d41b4f8b04d7cb80832c9b05c0f18" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41759 } health_report { overall_health: HEALTHY } } }
I20260619 06:50:38.117959 20628 raft_consensus.cc:1261] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Refusing update from remote peer 9a5d41b4f8b04d7cb80832c9b05c0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:38.118039 20486 raft_consensus.cc:1261] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Refusing update from remote peer 9a5d41b4f8b04d7cb80832c9b05c0f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:38.118587 20960 consensus_queue.cc:1035] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcfa021d06264f368f49a9ee88eb825f" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 39653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:38.119306 20960 consensus_queue.cc:1035] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab03559e33d1412ab413c145c960345f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:50:38.151317 20914 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 06:50:39.740857 20326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:39.741104 20326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:39.745237 20326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efe747fed9314d5fbef46070bef26601: Sending DeleteTablet for 3 replicas of tablet 82721e93de4b49b9a65bc1394b4359ee
I20260619 06:50:39.746136 20465 tablet_service.cc:1514] Processing DeleteTablet for tablet 82721e93de4b49b9a65bc1394b4359ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:39 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:48598
I20260619 06:50:39.746259 20605 tablet_service.cc:1514] Processing DeleteTablet for tablet 82721e93de4b49b9a65bc1394b4359ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:39 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60430
I20260619 06:50:39.746325 20744 tablet_service.cc:1514] Processing DeleteTablet for tablet 82721e93de4b49b9a65bc1394b4359ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:39 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53058
I20260619 06:50:39.747394 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20400
I20260619 06:50:39.762362 20993 tablet_replica.cc:330] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: stopping tablet replica
I20260619 06:50:39.762873 20993 raft_consensus.cc:2229] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:39.763397 20993 raft_consensus.cc:2258] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:39.765561 20993 ts_tablet_manager.cc:1905] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:39.776192 20993 ts_tablet_manager.cc:1918] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:50:39.776449 20993 log.cc:1196] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/82721e93de4b49b9a65bc1394b4359ee
I20260619 06:50:39.777038 20993 ts_tablet_manager.cc:1939] T 82721e93de4b49b9a65bc1394b4359ee P 9a5d41b4f8b04d7cb80832c9b05c0f18: Deleting consensus metadata
I20260619 06:50:39.778661 20314 catalog_manager.cc:4841] TS 9a5d41b4f8b04d7cb80832c9b05c0f18 (127.18.39.67:41759): tablet 82721e93de4b49b9a65bc1394b4359ee (table test-table [id=25fa2f16be5548229f8dbcc87f161969]) successfully deleted
I20260619 06:50:39.780031 20991 tablet_replica.cc:330] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: stopping tablet replica
I20260619 06:50:39.780352 20991 raft_consensus.cc:2229] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:39.780711 20991 raft_consensus.cc:2258] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:39.781405 20992 tablet_replica.cc:330] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: stopping tablet replica
I20260619 06:50:39.781638 20992 raft_consensus.cc:2229] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:39.781893 20992 raft_consensus.cc:2258] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:39.782701 20991 ts_tablet_manager.cc:1905] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:39.782737 20992 ts_tablet_manager.cc:1905] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:39.787919 20992 ts_tablet_manager.cc:1918] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
Jun 19 06:50:39 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:39.788079 20992 log.cc:1196] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/82721e93de4b49b9a65bc1394b4359ee
I20260619 06:50:39.788973 20992 ts_tablet_manager.cc:1939] T 82721e93de4b49b9a65bc1394b4359ee P dcfa021d06264f368f49a9ee88eb825f: Deleting consensus metadata
I20260619 06:50:39.788964 20991 ts_tablet_manager.cc:1918] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:50:39.789095 20991 log.cc:1196] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/82721e93de4b49b9a65bc1394b4359ee
I20260619 06:50:39.789602 20991 ts_tablet_manager.cc:1939] T 82721e93de4b49b9a65bc1394b4359ee P ab03559e33d1412ab413c145c960345f: Deleting consensus metadata
I20260619 06:50:39.790776 20314 catalog_manager.cc:4841] TS ab03559e33d1412ab413c145c960345f (127.18.39.65:41235): tablet 82721e93de4b49b9a65bc1394b4359ee (table test-table [id=25fa2f16be5548229f8dbcc87f161969]) successfully deleted
I20260619 06:50:39.791069 20314 catalog_manager.cc:4841] TS dcfa021d06264f368f49a9ee88eb825f (127.18.39.66:39653): tablet 82721e93de4b49b9a65bc1394b4359ee (table test-table [id=25fa2f16be5548229f8dbcc87f161969]) successfully deleted
I20260619 06:50:39.796066 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:39.750700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.18.39.65:41235)
0619 06:50:39.771037 (+ 20337us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:39.771046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:39.771081 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:50:39.771194 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:39.771202 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:39.771309 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:39.771464 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:39.774068 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.775645 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.779535 (+ 3890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.780186 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.780356 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:39.789525 (+ 9169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:39.789551 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:39.789577 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:39.789636 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:39.793117 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.793906 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.793915 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.793917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.794053 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.794470 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.794480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.794482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.794828 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:39.795145 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:39.795475 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 06:50:39.795708 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20149,"thread_start_us":25113,"threads_started":1}
W20260619 06:50:39.796846 20531 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39950
W20260619 06:50:39.797590 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20400: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:39.797832 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 20400
I20260619 06:50:39.823704 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20540
Jun 19 06:50:39 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:39.838842 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:39.824463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.18.39.66:39653)
0619 06:50:39.824965 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:39.824970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:39.824985 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:50:39.825038 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:39.825042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:39.825102 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:39.825190 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:39.826438 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.827169 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.828347 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.828829 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.828941 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:39.833211 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:39.833227 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:39.833244 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:39.833280 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:39.836372 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.836993 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.837000 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.837002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.837126 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.837490 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.837493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.837495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.837845 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:39.838086 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:39.838281 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 06:50:39.838443 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
W20260619 06:50:39.839599 20670 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45310
W20260619 06:50:39.840330 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20540: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:39.840492 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 20540
I20260619 06:50:39.862206 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20679
Jun 19 06:50:39 dist-test-slave-1h5s krb5kdc[20284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851837, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:39.877982 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:39.862986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.18.39.67:41759)
0619 06:50:39.863473 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:39.863477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:39.863493 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:50:39.863549 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:39.863553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:39.863615 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:39.863706 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:39.864990 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.865788 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.867148 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:39.867567 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:39.867663 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:39.872409 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:39.872427 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:39.872443 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:39.872481 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:39.875267 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.876066 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.876073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.876075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.876203 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:39.876525 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:39.876528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:39.876530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:39.876876 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:39.877097 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:39.877532 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 06:50:39.877708 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":260,"threads_started":1}
W20260619 06:50:39.878670 20809 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46316
W20260619 06:50:39.879261 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20679: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:39.879455 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 20679
I20260619 06:50:39.900871 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 20297
I20260619 06:50:39.902096 20369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:40.017017 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 20297
2026-06-19T06:50:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8633 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:40 dist-test-slave-1h5s krb5kdc[21001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:40 dist-test-slave-1h5s krb5kdc[21001](info): set up 2 sockets
Jun 19 06:50:40 dist-test-slave-1h5s krb5kdc[21001](info): commencing operation
krb5kdc: starting...
W20260619 06:50:41.105695 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.006s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:41 dist-test-slave-1h5s krb5kdc[21001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:41.296907 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46757
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:46757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:41.624431 21014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:41.624742 21014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:41.624927 21014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:41.632336 21014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:41.632464 21014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:41.632575 21014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:41.632682 21014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:41.632766 21014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:41.642881 21014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:46757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:41.644255 21014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:41.645682 21014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:41.657745 21020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:41.657836 21019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:41.657837 21022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:41.658365 21014 server_base.cc:1034] running on GCE node
I20260619 06:50:41.659175 21014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:41.660779 21014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:41.661972 21014 hybrid_clock.cc:648] HybridClock initialized: now 1781851841661926 us; error 33 us; skew 500 ppm
I20260619 06:50:41.662436 21014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:41 dist-test-slave-1h5s krb5kdc[21001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851841, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:41.665678 21014 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:41.667145 21014 webserver.cc:460] Webserver started at http://127.18.39.126:36591/ using document root <none> and password file <none>
I20260619 06:50:41.667865 21014 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:41.667994 21014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:41.668331 21014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:41.671451 21014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "7b59d923cc454d919a4e92976b47458a"
format_stamp: "Formatted at 2026-06-19 06:50:41 on dist-test-slave-1h5s"
server_key: "a511ed16b458a7c02445927ab4a8f67e"
server_key_iv: "87a62eeaf1378c1154652080ffa1875f"
server_key_version: "encryptionkey@0"
I20260619 06:50:41.672205 21014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "7b59d923cc454d919a4e92976b47458a"
format_stamp: "Formatted at 2026-06-19 06:50:41 on dist-test-slave-1h5s"
server_key: "a511ed16b458a7c02445927ab4a8f67e"
server_key_iv: "87a62eeaf1378c1154652080ffa1875f"
server_key_version: "encryptionkey@0"
I20260619 06:50:41.677604 21014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:50:41.681090 21029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:41.682477 21014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:41.682691 21014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "7b59d923cc454d919a4e92976b47458a"
format_stamp: "Formatted at 2026-06-19 06:50:41 on dist-test-slave-1h5s"
server_key: "a511ed16b458a7c02445927ab4a8f67e"
server_key_iv: "87a62eeaf1378c1154652080ffa1875f"
server_key_version: "encryptionkey@0"
I20260619 06:50:41.682906 21014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:41.711889 21014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:41.716130 21014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:41.716446 21014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:41.735075 21014 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:46757
I20260619 06:50:41.735164 21080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:46757 every 8 connection(s)
I20260619 06:50:41.736493 21014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:41.740190 21081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:41.745718 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21014
I20260619 06:50:41.745954 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:41.746573 18589 external_mini_cluster.cc:1428] Setting key 8f3bc73c9e728dea0e6fb8509e82dc54
I20260619 06:50:41.752208 21081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a: Bootstrap starting.
I20260619 06:50:41.755908 21081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:41.757176 21081 log.cc:824] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:41.760499 21081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a: No bootstrap required, opened a new log
Jun 19 06:50:41 dist-test-slave-1h5s krb5kdc[21001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:41.770596 21081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } }
I20260619 06:50:41.771143 21081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:41.771315 21081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b59d923cc454d919a4e92976b47458a, State: Initialized, Role: FOLLOWER
I20260619 06:50:41.772011 21081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } }
I20260619 06:50:41.772290 21081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:41.772469 21081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:41.772660 21081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:41.772799 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:41.748663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34726 (local address 127.18.39.126:46757)
0619 06:50:41.749816 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:41.749833 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:41.749881 (+ 48us) server_negotiation.cc:407] Connection header received
0619 06:50:41.752278 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:41.752309 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:41.752806 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:41.753247 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:41.754900 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:41.756871 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:41.758656 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:41.759635 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:41.759832 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:41.763703 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:41.763718 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:41.763739 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:41.763788 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:41.767963 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:41.768690 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:41.768705 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:41.768716 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:41.768961 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:41.769355 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:41.769360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:41.769365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:41.770120 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:41.770412 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:41.770813 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 06:50:41.771582 (+ 769us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":651,"threads_started":1}
I20260619 06:50:41.775560 21081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } }
I20260619 06:50:41.776206 21081 leader_election.cc:302] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b59d923cc454d919a4e92976b47458a; no voters:
I20260619 06:50:41.776742 21081 leader_election.cc:288] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:41.776855 21086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:41.777619 21086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [term 1 LEADER]: Becoming Leader. State: Replica: 7b59d923cc454d919a4e92976b47458a, State: Running, Role: LEADER
I20260619 06:50:41.778107 21086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } }
I20260619 06:50:41.778852 21081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:41.780941 21088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b59d923cc454d919a4e92976b47458a. Latest consensus state: current_term: 1 leader_uuid: "7b59d923cc454d919a4e92976b47458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } } }
I20260619 06:50:41.781208 21087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b59d923cc454d919a4e92976b47458a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b59d923cc454d919a4e92976b47458a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46757 } } }
I20260619 06:50:41.781497 21088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:41.782055 21087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:41.783849 21095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:41.788604 21095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:41.797726 21095 catalog_manager.cc:1347] Generated new cluster ID: 33058eb57f914b4487fc30c88b60b222
I20260619 06:50:41.797863 21095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:41.813517 21095 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:41.815440 21095 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:41.821913 21095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b59d923cc454d919a4e92976b47458a: Generated new TSK 0
I20260619 06:50:41.822975 21095 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:41.869123 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:42.203925 21107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:42.204299 21107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:42.204488 21107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:42.212105 21107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:42.212289 21107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:42.212473 21107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:42.222699 21107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:42.224121 21107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:42.225553 21107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:42.239058 21112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:42.239092 21113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:42.239058 21115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:42.240324 21107 server_base.cc:1034] running on GCE node
I20260619 06:50:42.240937 21107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:42.241920 21107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:42.243125 21107 hybrid_clock.cc:648] HybridClock initialized: now 1781851842243079 us; error 31 us; skew 500 ppm
I20260619 06:50:42.243489 21107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:42 dist-test-slave-1h5s krb5kdc[21001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851842, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:42.246690 21107 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:42.248116 21107 webserver.cc:460] Webserver started at http://127.18.39.65:39667/ using document root <none> and password file <none>
I20260619 06:50:42.248739 21107 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:42.248834 21107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:42.249084 21107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:42.251967 21107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "9b4fb97df7ff4a6db25f2dbc47eafdea"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "5f2e984bc73d73bbe58a618252b71859"
server_key_iv: "f3242008f7e588888b99d2833cf565db"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.252794 21107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "9b4fb97df7ff4a6db25f2dbc47eafdea"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "5f2e984bc73d73bbe58a618252b71859"
server_key_iv: "f3242008f7e588888b99d2833cf565db"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.257946 21107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260619 06:50:42.261304 21122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.262691 21107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:42.262846 21107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "9b4fb97df7ff4a6db25f2dbc47eafdea"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "5f2e984bc73d73bbe58a618252b71859"
server_key_iv: "f3242008f7e588888b99d2833cf565db"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.263049 21107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:42.277521 21107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:42.282112 21107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:42.282410 21107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:42.283514 21107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:42.284847 21107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:42.284917 21107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.285006 21107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:42.285054 21107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.317780 21107 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:36041
I20260619 06:50:42.317818 21234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:36041 every 8 connection(s)
I20260619 06:50:42.319056 21107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:42.319779 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21107
I20260619 06:50:42.320055 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:42.320883 18589 external_mini_cluster.cc:1428] Setting key 7504b261ed175991cfa04ba8789d3273
Jun 19 06:50:42 dist-test-slave-1h5s krb5kdc[21001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851842, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:50:42.341313 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:42.321905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:47229 (local address 127.18.39.126:46757)
0619 06:50:42.322389 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:42.322393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:42.323497 (+ 1104us) server_negotiation.cc:407] Connection header received
0619 06:50:42.325511 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:42.325516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:42.325614 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:42.325731 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:42.327205 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.328371 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.330041 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.330821 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.330972 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:42.334580 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:42.334589 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:42.334598 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:42.334645 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:42.338280 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:42.339034 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:42.339042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:42.339048 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:42.339197 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:42.339653 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:42.339660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:42.339663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:42.340015 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:42.340135 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:42.340866 (+ 731us) server_negotiation.cc:299] Negotiation successful
0619 06:50:42.341039 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":850,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:50:42.342366 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:42.322479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46757 (local address 127.18.39.65:47229)
0619 06:50:42.323340 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:42.323384 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:42.325179 (+ 1795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:42.325972 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:42.325981 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:42.326480 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:42.327035 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:42.327053 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.328550 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.328554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:42.329879 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:42.329892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.331023 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.331028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:42.331238 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:42.331926 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:42.331947 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:42.334345 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:42.338435 (+ 4090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:42.338442 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:42.338456 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:42.338884 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:42.339309 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:42.339312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:42.339315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:42.339490 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:42.340153 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:42.340159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:42.340517 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 06:50:42.340761 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 06:50:42.341149 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":327,"threads_started":1}
I20260619 06:50:42.344116 21235 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46757
I20260619 06:50:42.344434 21235 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:42.345165 21235 heartbeater.cc:511] Master 127.18.39.126:46757 requested a full tablet report, sending...
I20260619 06:50:42.347429 21046 ts_manager.cc:194] Registered new tserver with Master: 9b4fb97df7ff4a6db25f2dbc47eafdea (127.18.39.65:36041)
I20260619 06:50:42.349706 21046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:47229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:42.366840 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:42.700971 21241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:42.701284 21241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:42.701463 21241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:42.708807 21241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:42.708920 21241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:42.709035 21241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:42.718948 21241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:42.720197 21241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:42.721657 21241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:42.734807 21246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:42.734845 21247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:42.736027 21249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:42.737457 21241 server_base.cc:1034] running on GCE node
I20260619 06:50:42.738189 21241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:42.739343 21241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:42.740594 21241 hybrid_clock.cc:648] HybridClock initialized: now 1781851842740542 us; error 43 us; skew 500 ppm
I20260619 06:50:42.740947 21241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:42 dist-test-slave-1h5s krb5kdc[21001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851842, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:42.744824 21241 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:42.752799 21241 webserver.cc:460] Webserver started at http://127.18.39.66:40593/ using document root <none> and password file <none>
I20260619 06:50:42.753727 21241 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:42.753885 21241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:42.754204 21241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:42.757295 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "cc5810151af3478bba801680631fd4b7"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "885d7705deb2b7b00b82adad9b31379f"
server_key_iv: "80ad1566e8b878ce1bd710e328f0ede5"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.758213 21241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "cc5810151af3478bba801680631fd4b7"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "885d7705deb2b7b00b82adad9b31379f"
server_key_iv: "80ad1566e8b878ce1bd710e328f0ede5"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.763662 21241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:42.767583 21256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.768996 21241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:42.769207 21241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "cc5810151af3478bba801680631fd4b7"
format_stamp: "Formatted at 2026-06-19 06:50:42 on dist-test-slave-1h5s"
server_key: "885d7705deb2b7b00b82adad9b31379f"
server_key_iv: "80ad1566e8b878ce1bd710e328f0ede5"
server_key_version: "encryptionkey@0"
I20260619 06:50:42.769495 21241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:42.789737 21241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:42.794329 21241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:42.794669 21241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:42.795663 21241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:42.797201 21241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:42.797309 21241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.797488 21241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:42.797569 21241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:42.831310 21241 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:43125
I20260619 06:50:42.831353 21368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:43125 every 8 connection(s)
I20260619 06:50:42.832659 21241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:42.837474 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21241
I20260619 06:50:42.837674 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:42.838107 18589 external_mini_cluster.cc:1428] Setting key a2775d2ff4989d9a21a88787b11b1db5
Jun 19 06:50:42 dist-test-slave-1h5s krb5kdc[21001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851842, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:42.855192 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:42.835329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:37521 (local address 127.18.39.126:46757)
0619 06:50:42.835536 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:42.835540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:42.836676 (+ 1136us) server_negotiation.cc:407] Connection header received
0619 06:50:42.838835 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:42.838842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:42.838932 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:42.839019 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:42.840634 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.841390 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.843035 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.843765 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.843927 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:42.848062 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:42.848072 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:42.848080 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:42.848127 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:42.851747 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:42.852506 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:42.852514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:42.852519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:42.852694 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:42.853125 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:42.853133 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:42.853139 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:42.853572 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:42.853741 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:42.854556 (+ 815us) server_negotiation.cc:299] Negotiation successful
0619 06:50:42.854806 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 06:50:42.855990 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:42.835757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46757 (local address 127.18.39.66:37521)
0619 06:50:42.836510 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:42.836550 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:42.838501 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:42.839289 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:42.839298 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:42.839905 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:42.840467 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:42.840483 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.841545 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.841550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:42.842860 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:42.842876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:42.844022 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:42.844028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:42.844236 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:42.844989 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:42.845016 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:42.847803 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:42.851910 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:42.851928 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:42.851943 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:42.852364 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:42.852815 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:42.852818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:42.852820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:42.853008 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:42.853695 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:42.853701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:42.854178 (+ 477us) client_negotiation.cc:769] Sending connection context
0619 06:50:42.854424 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 06:50:42.854846 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":289,"threads_started":1}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:42.857581 21369 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46757
I20260619 06:50:42.857870 21369 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:42.858588 21369 heartbeater.cc:511] Master 127.18.39.126:46757 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:50:42.860265 21046 ts_manager.cc:194] Registered new tserver with Master: cc5810151af3478bba801680631fd4b7 (127.18.39.66:43125)
I20260619 06:50:42.861553 21046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:37521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:42.884508 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:43.214767 21374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:43.215123 21374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:43.215296 21374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:43.222589 21374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:43.222764 21374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:43.222958 21374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:43.238310 21374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:43.239741 21374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:43.241070 21374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:43.254520 21380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:43.254598 21379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:43.255016 21382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:43.255371 21374 server_base.cc:1034] running on GCE node
I20260619 06:50:43.256008 21374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:43.256984 21374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:43.258237 21374 hybrid_clock.cc:648] HybridClock initialized: now 1781851843258162 us; error 55 us; skew 500 ppm
I20260619 06:50:43.258602 21374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:43 dist-test-slave-1h5s krb5kdc[21001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851843, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:43.262420 21374 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:43.263895 21374 webserver.cc:460] Webserver started at http://127.18.39.67:33275/ using document root <none> and password file <none>
I20260619 06:50:43.264616 21374 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:43.264715 21374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:43.265039 21374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:43.268270 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "9d209d5a288148a09b1a153946cd92e1"
format_stamp: "Formatted at 2026-06-19 06:50:43 on dist-test-slave-1h5s"
server_key: "f6536a3526279ab8e2203de8ec278643"
server_key_iv: "0cdb0f2928ed49d4cc73f3dd8a5b2986"
server_key_version: "encryptionkey@0"
I20260619 06:50:43.269038 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "9d209d5a288148a09b1a153946cd92e1"
format_stamp: "Formatted at 2026-06-19 06:50:43 on dist-test-slave-1h5s"
server_key: "f6536a3526279ab8e2203de8ec278643"
server_key_iv: "0cdb0f2928ed49d4cc73f3dd8a5b2986"
server_key_version: "encryptionkey@0"
I20260619 06:50:43.274545 21374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:50:43.278198 21389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:43.279667 21374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:43.279857 21374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "9d209d5a288148a09b1a153946cd92e1"
format_stamp: "Formatted at 2026-06-19 06:50:43 on dist-test-slave-1h5s"
server_key: "f6536a3526279ab8e2203de8ec278643"
server_key_iv: "0cdb0f2928ed49d4cc73f3dd8a5b2986"
server_key_version: "encryptionkey@0"
I20260619 06:50:43.280059 21374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:43.300690 21374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:43.305095 21374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:43.305506 21374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:43.306515 21374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:43.307919 21374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:43.308032 21374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:43.308192 21374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:43.308267 21374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:43.342092 21374 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:39525
I20260619 06:50:43.342147 21501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:39525 every 8 connection(s)
I20260619 06:50:43.343542 21374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:43.352810 21235 heartbeater.cc:503] Master 127.18.39.126:46757 was elected leader, sending a full tablet report...
I20260619 06:50:43.353513 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21374
I20260619 06:50:43.353734 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:43.354247 18589 external_mini_cluster.cc:1428] Setting key dc79401f0c0db092c80a17c2c60dac69
Jun 19 06:50:43 dist-test-slave-1h5s krb5kdc[21001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851843, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:43.363651 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:43.346276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:51005 (local address 127.18.39.126:46757)
0619 06:50:43.346481 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:43.346485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:43.347661 (+ 1176us) server_negotiation.cc:407] Connection header received
0619 06:50:43.349765 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:43.349772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:43.349864 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:43.349981 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:43.351527 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:43.352293 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:43.354230 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:43.354858 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:43.354993 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:43.358471 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:43.358476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:43.358479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:43.358509 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:43.360874 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:43.361653 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:43.361657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:43.361659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:43.361775 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:43.362142 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:43.362145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:43.362147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:43.362446 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:43.362555 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:43.363222 (+ 667us) server_negotiation.cc:299] Negotiation successful
0619 06:50:43.363404 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:50:43.364631 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:43.346766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46757 (local address 127.18.39.67:51005)
0619 06:50:43.347481 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:43.347525 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:43.349465 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:43.350242 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:43.350250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:43.350758 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:43.351354 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:43.351371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:43.352443 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:43.352453 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:43.354086 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:43.354097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:43.355088 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:43.355096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:43.355334 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:43.356058 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:43.356102 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:43.358288 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:43.360997 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:43.361003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:43.361016 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:43.361547 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:43.361867 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:43.361870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:43.361872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:43.362046 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:43.362567 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:43.362573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:43.362893 (+ 320us) client_negotiation.cc:769] Sending connection context
0619 06:50:43.363125 (+ 232us) client_negotiation.cc:240] Negotiation successful
0619 06:50:43.363515 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":274,"threads_started":1}
I20260619 06:50:43.366073 21502 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46757
I20260619 06:50:43.366392 21502 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:43.367233 21502 heartbeater.cc:511] Master 127.18.39.126:46757 requested a full tablet report, sending...
I20260619 06:50:43.368764 21045 ts_manager.cc:194] Registered new tserver with Master: 9d209d5a288148a09b1a153946cd92e1 (127.18.39.67:39525)
I20260619 06:50:43.369797 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:43.369805 21045 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:51005
W20260619 06:50:43.400879 21237 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:43.398331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34728 (local address 127.18.39.126:46757)
0619 06:50:43.398527 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:43.398532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:43.399020 (+ 488us) server_negotiation.cc:407] Connection header received
0619 06:50:43.399240 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:43.399245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:43.399315 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:43.399415 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:43.400611 (+ 1196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34728: BlockingRecv error: recv got EOF from 127.0.0.1:34728 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:50:43.404039 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21107
W20260619 06:50:43.406888 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21107: Not authorized: Client connection negotiation failed: client connection to 127.18.39.65:36041: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:50:43.407078 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21107
I20260619 06:50:43.422935 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21241
W20260619 06:50:43.425746 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21241: Not authorized: Client connection negotiation failed: client connection to 127.18.39.66:43125: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:50:43.425941 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21241
I20260619 06:50:43.440956 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21374
W20260619 06:50:43.443893 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21374: Not authorized: Client connection negotiation failed: client connection to 127.18.39.67:39525: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:50:43.444053 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21374
I20260619 06:50:43.459617 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21014
I20260619 06:50:43.460953 21076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:43.563040 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21014
2026-06-19T06:50:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3543 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:43 dist-test-slave-1h5s krb5kdc[21519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:43 dist-test-slave-1h5s krb5kdc[21519](info): set up 2 sockets
Jun 19 06:50:43 dist-test-slave-1h5s krb5kdc[21519](info): commencing operation
krb5kdc: starting...
W20260619 06:50:44.665753 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.008s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:44 dist-test-slave-1h5s krb5kdc[21519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:44.857412 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34141
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:33793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:45.184767 21532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:45.185141 21532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:45.185310 21532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:45.192736 21532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:45.192878 21532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:45.192924 21532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:45.192977 21532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:45.193027 21532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:45.203156 21532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:33793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:45.204432 21532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:45.205767 21532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:45.217845 21538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:45.217800 21537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:45.217895 21540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:45.223116 21532 server_base.cc:1034] running on GCE node
I20260619 06:50:45.223987 21532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:45.225513 21532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:45.226723 21532 hybrid_clock.cc:648] HybridClock initialized: now 1781851845226684 us; error 41 us; skew 500 ppm
I20260619 06:50:45.227097 21532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:45 dist-test-slave-1h5s krb5kdc[21519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851845, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:45.235776 21532 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:45.237567 21532 webserver.cc:460] Webserver started at http://127.18.39.126:33399/ using document root <none> and password file <none>
I20260619 06:50:45.238348 21532 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:45.238509 21532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:45.238839 21532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:45.241998 21532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "5d524411eb0643ad86b1a75dc6f6402e"
format_stamp: "Formatted at 2026-06-19 06:50:45 on dist-test-slave-1h5s"
server_key: "36bb796e57002207325ee3b4add552d9"
server_key_iv: "797f9a1a6e8bfc445f66d222ae6bc89a"
server_key_version: "encryptionkey@0"
I20260619 06:50:45.242817 21532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "5d524411eb0643ad86b1a75dc6f6402e"
format_stamp: "Formatted at 2026-06-19 06:50:45 on dist-test-slave-1h5s"
server_key: "36bb796e57002207325ee3b4add552d9"
server_key_iv: "797f9a1a6e8bfc445f66d222ae6bc89a"
server_key_version: "encryptionkey@0"
I20260619 06:50:45.248167 21532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:45.251857 21547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:45.253276 21532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:50:45.253535 21532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "5d524411eb0643ad86b1a75dc6f6402e"
format_stamp: "Formatted at 2026-06-19 06:50:45 on dist-test-slave-1h5s"
server_key: "36bb796e57002207325ee3b4add552d9"
server_key_iv: "797f9a1a6e8bfc445f66d222ae6bc89a"
server_key_version: "encryptionkey@0"
I20260619 06:50:45.253755 21532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:45.288468 21532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:45.292840 21532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:45.293169 21532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:45.312127 21532 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:34141
I20260619 06:50:45.312166 21598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:34141 every 8 connection(s)
I20260619 06:50:45.313635 21532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:45.316431 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21532
I20260619 06:50:45.316666 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:45.317488 18589 external_mini_cluster.cc:1428] Setting key 1c9153447d2a082d1874c99e87ff78f3
I20260619 06:50:45.318043 21599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:45.332741 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e: Bootstrap starting.
Jun 19 06:50:45 dist-test-slave-1h5s krb5kdc[21519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:45.336903 21599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:45.338536 21599 log.cc:824] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:45.342259 21599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e: No bootstrap required, opened a new log
I20260619 06:50:45.342989 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:45.319397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.18.39.126:34141)
0619 06:50:45.320254 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:45.320266 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:45.320306 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:50:45.322666 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:45.322691 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:45.323137 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:45.323499 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:45.325466 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:45.326753 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:45.328463 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:45.329167 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:45.329303 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:45.333941 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:45.333952 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:45.333971 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:45.334016 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:45.338098 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:45.338845 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:45.338856 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:45.338869 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:45.339096 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:45.339582 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:45.339586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:45.339589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:45.340328 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:45.340643 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:45.341017 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 06:50:45.341520 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":389,"threads_started":1}
I20260619 06:50:45.346984 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:45.351092 21599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } }
I20260619 06:50:45.351544 21599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:45.351677 21599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d524411eb0643ad86b1a75dc6f6402e, State: Initialized, Role: FOLLOWER
I20260619 06:50:45.352335 21599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } }
I20260619 06:50:45.352658 21599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:45.352823 21599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:45.353014 21599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:45.355473 21599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } }
I20260619 06:50:45.356040 21599 leader_election.cc:302] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d524411eb0643ad86b1a75dc6f6402e; no voters:
I20260619 06:50:45.356742 21599 leader_election.cc:288] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:45.356894 21605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:45.359695 21599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:45.361025 21605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [term 1 LEADER]: Becoming Leader. State: Replica: 5d524411eb0643ad86b1a75dc6f6402e, State: Running, Role: LEADER
I20260619 06:50:45.361861 21605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } }
I20260619 06:50:45.365729 21607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d524411eb0643ad86b1a75dc6f6402e. Latest consensus state: current_term: 1 leader_uuid: "5d524411eb0643ad86b1a75dc6f6402e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } } }
I20260619 06:50:45.366282 21607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:45.367039 21606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d524411eb0643ad86b1a75dc6f6402e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d524411eb0643ad86b1a75dc6f6402e" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34141 } } }
I20260619 06:50:45.367324 21606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:45.367388 21617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:45.382094 21617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:45.394897 21617 catalog_manager.cc:1347] Generated new cluster ID: cf0f75423b9b412589da62dfa3db5eee
I20260619 06:50:45.395036 21617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:45.416160 21617 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:45.417734 21617 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:45.425251 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:45.414179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.18.39.126:34141)
0619 06:50:45.414410 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:45.414420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:45.415382 (+ 962us) server_negotiation.cc:407] Connection header received
0619 06:50:45.416473 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:45.416478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:45.416568 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:45.416694 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:45.418169 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:45.419307 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:45.421406 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:45.422073 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:45.422187 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:45.423508 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:45.423515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:45.423520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:45.424357 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:45.424413 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:45.424452 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:45.424552 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:45.424733 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 06:50:45.424859 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":95}
I20260619 06:50:45.431161 21617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d524411eb0643ad86b1a75dc6f6402e: Generated new TSK 0
I20260619 06:50:45.432308 21617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:45.445482 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21532
I20260619 06:50:45.446802 21594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:45.540377 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21532
2026-06-19T06:50:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1971 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:45 dist-test-slave-1h5s krb5kdc[21634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:45 dist-test-slave-1h5s krb5kdc[21634](info): set up 2 sockets
Jun 19 06:50:45 dist-test-slave-1h5s krb5kdc[21634](info): commencing operation
krb5kdc: starting...
W20260619 06:50:46.625702 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:46 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:46.820987 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34289
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:34289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:47.146283 21647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:47.146592 21647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:47.146747 21647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:47.154503 21647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:47.154618 21647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:47.154731 21647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:47.154810 21647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:47.154872 21647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:47.164593 21647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:34289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:47.165907 21647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:47.167201 21647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:47.178851 21652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:47.179028 21655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:47.179145 21653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:47.180166 21647 server_base.cc:1034] running on GCE node
I20260619 06:50:47.181069 21647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:47.182575 21647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:47.183816 21647 hybrid_clock.cc:648] HybridClock initialized: now 1781851847183745 us; error 66 us; skew 500 ppm
I20260619 06:50:47.184178 21647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:47 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851847, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:47.187348 21647 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:47.188992 21647 webserver.cc:460] Webserver started at http://127.18.39.126:34597/ using document root <none> and password file <none>
I20260619 06:50:47.189750 21647 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:47.189893 21647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:47.190243 21647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:47.193213 21647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "009cd23422e9471bba31617dbe112847"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "5154675d8f67410949df13b125ed52ab"
server_key_iv: "87e6fdce61f1d5df8dc168c42ad8eac8"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.193998 21647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "009cd23422e9471bba31617dbe112847"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "5154675d8f67410949df13b125ed52ab"
server_key_iv: "87e6fdce61f1d5df8dc168c42ad8eac8"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.199261 21647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:50:47.202728 21662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:47.204140 21647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:47.204334 21647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "009cd23422e9471bba31617dbe112847"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "5154675d8f67410949df13b125ed52ab"
server_key_iv: "87e6fdce61f1d5df8dc168c42ad8eac8"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.204546 21647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:47.236706 21647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:47.241016 21647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:47.241351 21647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:47.260023 21647 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:34289
I20260619 06:50:47.260046 21713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:34289 every 8 connection(s)
I20260619 06:50:47.261514 21647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:47.265242 21714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:47.270783 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21647
I20260619 06:50:47.271008 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:47.271667 18589 external_mini_cluster.cc:1428] Setting key 7b7e4d77a54d6b2363f5399b0fc77881
I20260619 06:50:47.277356 21714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847: Bootstrap starting.
I20260619 06:50:47.281057 21714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:47.282224 21714 log.cc:824] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:47.285490 21714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847: No bootstrap required, opened a new log
Jun 19 06:50:47 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:47.295815 21714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } }
I20260619 06:50:47.296283 21714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:47.296386 21714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 009cd23422e9471bba31617dbe112847, State: Initialized, Role: FOLLOWER
I20260619 06:50:47.297004 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:47.273874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.18.39.126:34289)
0619 06:50:47.274722 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:47.274749 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:47.274804 (+ 55us) server_negotiation.cc:407] Connection header received
0619 06:50:47.277143 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:47.277174 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:47.277745 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:47.278183 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:47.279507 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.281489 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.282964 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.283779 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.283953 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:47.288020 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:47.288030 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:47.288051 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:47.288109 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:47.292257 (+ 4148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:47.292993 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:47.293002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:47.293011 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:47.293230 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:47.293655 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:47.293660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:47.293664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:47.294489 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:47.294803 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:47.295167 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 06:50:47.295623 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":349,"threads_started":1}
I20260619 06:50:47.297079 21714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } }
I20260619 06:50:47.297370 21714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:47.297544 21714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:47.297740 21714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:47.300113 21714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } }
I20260619 06:50:47.300624 21714 leader_election.cc:302] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 009cd23422e9471bba31617dbe112847; no voters:
I20260619 06:50:47.301196 21714 leader_election.cc:288] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:47.301316 21719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:47.302647 21719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [term 1 LEADER]: Becoming Leader. State: Replica: 009cd23422e9471bba31617dbe112847, State: Running, Role: LEADER
I20260619 06:50:47.303189 21719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } }
I20260619 06:50:47.303361 21714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:47.306494 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 009cd23422e9471bba31617dbe112847. Latest consensus state: current_term: 1 leader_uuid: "009cd23422e9471bba31617dbe112847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } } }
I20260619 06:50:47.306540 21720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "009cd23422e9471bba31617dbe112847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "009cd23422e9471bba31617dbe112847" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34289 } } }
I20260619 06:50:47.306908 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:47.306964 21720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:47.307838 21725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:47.313966 21725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:47.323096 21725 catalog_manager.cc:1347] Generated new cluster ID: 20c848d9c7c04ee9a5683afe91f9bb17
I20260619 06:50:47.323235 21725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:47.336047 21725 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:47.337950 21725 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:47.345988 21725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 009cd23422e9471bba31617dbe112847: Generated new TSK 0
I20260619 06:50:47.347074 21725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:47.403383 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:47.738795 21740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:47.739136 21740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:47.739308 21740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:47.746667 21740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:47.746832 21740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:47.747023 21740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:47.757014 21740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:47.758420 21740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:47.759752 21740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:47.773133 21745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:47.773244 21746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:47.773130 21748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:47.774385 21740 server_base.cc:1034] running on GCE node
I20260619 06:50:47.775049 21740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:47.775955 21740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:47.777153 21740 hybrid_clock.cc:648] HybridClock initialized: now 1781851847777118 us; error 31 us; skew 500 ppm
I20260619 06:50:47.777490 21740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:47 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851847, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:47.780443 21740 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:47.781940 21740 webserver.cc:460] Webserver started at http://127.18.39.65:44059/ using document root <none> and password file <none>
I20260619 06:50:47.782609 21740 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:47.782752 21740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:47.783041 21740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:47.786015 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "d83d006c87d140ee88e312e6f23ff69d"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "e239cba1a3d5f612e115c4ce4ab0363b"
server_key_iv: "ece23f552b14bcf407ff7e15e68f0e6e"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.786824 21740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "d83d006c87d140ee88e312e6f23ff69d"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "e239cba1a3d5f612e115c4ce4ab0363b"
server_key_iv: "ece23f552b14bcf407ff7e15e68f0e6e"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.792302 21740 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:47.795810 21755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:47.797214 21740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 06:50:47.797427 21740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "d83d006c87d140ee88e312e6f23ff69d"
format_stamp: "Formatted at 2026-06-19 06:50:47 on dist-test-slave-1h5s"
server_key: "e239cba1a3d5f612e115c4ce4ab0363b"
server_key_iv: "ece23f552b14bcf407ff7e15e68f0e6e"
server_key_version: "encryptionkey@0"
I20260619 06:50:47.797616 21740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:47.809748 21740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:47.813979 21740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:47.814271 21740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:47.815215 21740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:47.816506 21740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:47.816582 21740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:47.816751 21740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:47.816810 21740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:47.849893 21740 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:38533
I20260619 06:50:47.849941 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:38533 every 8 connection(s)
I20260619 06:50:47.851233 21740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:47.854264 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21740
I20260619 06:50:47.854463 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:47.855253 18589 external_mini_cluster.cc:1428] Setting key c813e18b89ffdc38cb3feee4609a1c11
Jun 19 06:50:47 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851847, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:47.875671 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:47.853903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:50867 (local address 127.18.39.126:34289)
0619 06:50:47.854463 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:47.854469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:47.855607 (+ 1138us) server_negotiation.cc:407] Connection header received
0619 06:50:47.857818 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:47.857822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:47.857893 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:47.857983 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:47.860299 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.861003 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.863539 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.864252 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.864419 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:47.868235 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:47.868241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:47.868247 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:47.868296 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:47.871979 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:47.872880 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:47.872887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:47.872890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:47.873068 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:47.873622 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:47.873630 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:47.873635 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:47.874021 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:47.874181 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:47.874940 (+ 759us) server_negotiation.cc:299] Negotiation successful
0619 06:50:47.875212 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":246,"threads_started":1}
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:50:47.876499 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:47.854517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34289 (local address 127.18.39.65:50867)
0619 06:50:47.855413 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:47.855463 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:47.857524 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:47.858365 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:47.858382 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:47.859228 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:47.860112 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:47.860144 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.861169 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.861178 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:47.863348 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:47.863361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:47.864498 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:47.864503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:47.864712 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:47.865522 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:47.865544 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:47.867964 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:47.872170 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:47.872178 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:47.872195 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:47.872697 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:47.873198 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:47.873202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:47.873204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:47.873491 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:47.874158 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:47.874165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:47.874531 (+ 366us) client_negotiation.cc:769] Sending connection context
0619 06:50:47.874777 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 06:50:47.875209 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":305,"threads_started":1}
I20260619 06:50:47.878244 21868 heartbeater.cc:347] Connected to a master server at 127.18.39.126:34289
I20260619 06:50:47.878584 21868 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:50:47.879370 21868 heartbeater.cc:511] Master 127.18.39.126:34289 requested a full tablet report, sending...
I20260619 06:50:47.881834 21679 ts_manager.cc:194] Registered new tserver with Master: d83d006c87d140ee88e312e6f23ff69d (127.18.39.65:38533)
I20260619 06:50:47.883260 21679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:50867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:47.904307 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:48.233238 21874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:48.233568 21874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:48.233664 21874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:48.242141 21874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:48.242316 21874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:48.242524 21874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:48.255137 21874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:48.256517 21874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:48.257910 21874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:48.274120 21880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:48.274525 21879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:48.274919 21874 server_base.cc:1034] running on GCE node
W20260619 06:50:48.275166 21882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:48.275885 21874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:48.276928 21874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:48.278129 21874 hybrid_clock.cc:648] HybridClock initialized: now 1781851848278080 us; error 40 us; skew 500 ppm
I20260619 06:50:48.278523 21874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:48.281728 21874 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:48.283213 21874 webserver.cc:460] Webserver started at http://127.18.39.66:33993/ using document root <none> and password file <none>
I20260619 06:50:48.283885 21874 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:48.283985 21874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:48.284245 21874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:48.287202 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "1721bbe673494e1fb5e2f41487c7536a"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "0299f778c853fa85b3a1bd60f79f9c6d"
server_key_iv: "8a63f810ed33f5ce781513f99e0edef4"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.287978 21874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "1721bbe673494e1fb5e2f41487c7536a"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "0299f778c853fa85b3a1bd60f79f9c6d"
server_key_iv: "8a63f810ed33f5ce781513f99e0edef4"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.293680 21874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 06:50:48.297191 21889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.298662 21874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:48.298854 21874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "1721bbe673494e1fb5e2f41487c7536a"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "0299f778c853fa85b3a1bd60f79f9c6d"
server_key_iv: "8a63f810ed33f5ce781513f99e0edef4"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.299063 21874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:48.321858 21874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:48.326342 21874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:48.326648 21874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:48.327761 21874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:48.329615 21874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:48.329721 21874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.329844 21874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:48.329921 21874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.365237 21874 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:36641
I20260619 06:50:48.365386 22001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:36641 every 8 connection(s)
I20260619 06:50:48.366565 21874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:48.374122 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 21874
I20260619 06:50:48.374325 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:48.374898 18589 external_mini_cluster.cc:1428] Setting key 28b3dd52e279d0af998b974addb5b647
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:48.388048 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.369243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:55385 (local address 127.18.39.126:34289)
0619 06:50:48.369502 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:48.369508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:48.370644 (+ 1136us) server_negotiation.cc:407] Connection header received
0619 06:50:48.372371 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:48.372375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:48.372444 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:48.372537 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:48.374081 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.375216 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.377206 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.377889 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.378048 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.381755 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:48.381764 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:48.381772 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:48.381821 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:48.384538 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.385567 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.385572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.385574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.385717 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.386213 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.386216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.386219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.386491 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:48.386599 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:48.387582 (+ 983us) server_negotiation.cc:299] Negotiation successful
0619 06:50:48.387779 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 06:50:48.389637 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.369733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34289 (local address 127.18.39.66:55385)
0619 06:50:48.370451 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:48.370493 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:48.372073 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:48.372753 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:48.372761 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:48.373250 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:48.373901 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:48.373917 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.375421 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.375427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:48.377047 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:48.377061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.378089 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.378094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:48.378288 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.379001 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:48.379024 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:48.381515 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:48.384691 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:48.384703 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:48.384727 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:48.385435 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:48.385838 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:48.385846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:48.385852 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:48.386100 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:48.386619 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:48.386630 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:48.387134 (+ 504us) client_negotiation.cc:769] Sending connection context
0619 06:50:48.387441 (+ 307us) client_negotiation.cc:240] Negotiation successful
0619 06:50:48.388070 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":274,"threads_started":1}
I20260619 06:50:48.391366 22002 heartbeater.cc:347] Connected to a master server at 127.18.39.126:34289
I20260619 06:50:48.391696 22002 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:48.392436 22002 heartbeater.cc:511] Master 127.18.39.126:34289 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:48.394019 21679 ts_manager.cc:194] Registered new tserver with Master: 1721bbe673494e1fb5e2f41487c7536a (127.18.39.66:36641)
I20260619 06:50:48.394804 21679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:55385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:48.420616 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:48.751008 22007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:48.751407 22007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:48.751580 22007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:48.759063 22007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:48.759243 22007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:48.759434 22007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:48.769784 22007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:48.771127 22007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:48.772475 22007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:48.786374 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:48.786387 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:48.786435 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:48.787480 22007 server_base.cc:1034] running on GCE node
I20260619 06:50:48.788053 22007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:48.788961 22007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:48.790187 22007 hybrid_clock.cc:648] HybridClock initialized: now 1781851848790144 us; error 57 us; skew 500 ppm
I20260619 06:50:48.790537 22007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:48.793843 22007 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:48.795429 22007 webserver.cc:460] Webserver started at http://127.18.39.67:34019/ using document root <none> and password file <none>
I20260619 06:50:48.796145 22007 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:48.796294 22007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:48.796684 22007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:48.799769 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "24b875fef6ec46d0a67d2fa542756cf2"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "8fc821895ef7d724a6f5c302c8387250"
server_key_iv: "39c8b2a46415a85bd6a22be3925f10c7"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.800562 22007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "24b875fef6ec46d0a67d2fa542756cf2"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "8fc821895ef7d724a6f5c302c8387250"
server_key_iv: "39c8b2a46415a85bd6a22be3925f10c7"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.806514 22007 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260619 06:50:48.810472 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.811834 22007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:50:48.812009 22007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "24b875fef6ec46d0a67d2fa542756cf2"
format_stamp: "Formatted at 2026-06-19 06:50:48 on dist-test-slave-1h5s"
server_key: "8fc821895ef7d724a6f5c302c8387250"
server_key_iv: "39c8b2a46415a85bd6a22be3925f10c7"
server_key_version: "encryptionkey@0"
I20260619 06:50:48.812215 22007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:48.847005 22007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:48.851615 22007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:48.851938 22007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:48.852917 22007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:48.854368 22007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:48.854473 22007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.854625 22007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:48.854696 22007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:48.886809 21868 heartbeater.cc:503] Master 127.18.39.126:34289 was elected leader, sending a full tablet report...
I20260619 06:50:48.889182 22007 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:41153
I20260619 06:50:48.889225 22134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:41153 every 8 connection(s)
I20260619 06:50:48.890656 22007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:48.900542 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22007
I20260619 06:50:48.900727 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:48.901301 18589 external_mini_cluster.cc:1428] Setting key a5e20ba374ddfd0e8cdfe928e212587a
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:48.912026 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.893237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:58715 (local address 127.18.39.126:34289)
0619 06:50:48.893709 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:48.893715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:48.894933 (+ 1218us) server_negotiation.cc:407] Connection header received
0619 06:50:48.896790 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:48.896795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:48.896890 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:48.897010 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:48.898602 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.899370 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.901083 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.901641 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.901791 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.906008 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:48.906013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:48.906016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:48.906049 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:48.908714 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.909603 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.909615 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.909618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.909732 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.910175 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.910180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.910183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.910547 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:48.910704 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:48.911433 (+ 729us) server_negotiation.cc:299] Negotiation successful
0619 06:50:48.911649 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":253,"threads_started":1}
I20260619 06:50:48.913031 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.894015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34289 (local address 127.18.39.67:58715)
0619 06:50:48.894747 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:48.894795 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:48.896515 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:48.897245 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:48.897253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:48.897853 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:48.898423 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:48.898441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.899519 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.899532 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:48.900938 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:48.900949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.901888 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.901896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:48.902167 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.903028 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:48.903075 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:48.905771 (+ 2696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:48.908850 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:48.908858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:48.908872 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:48.909476 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:48.909850 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:48.909855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:48.909859 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:48.910049 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:48.910679 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:48.910687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:48.911047 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 06:50:48.911319 (+ 272us) client_negotiation.cc:240] Negotiation successful
0619 06:50:48.911885 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":320,"threads_started":1}
I20260619 06:50:48.914498 22135 heartbeater.cc:347] Connected to a master server at 127.18.39.126:34289
I20260619 06:50:48.914834 22135 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:48.915807 22135 heartbeater.cc:511] Master 127.18.39.126:34289 requested a full tablet report, sending...
I20260619 06:50:48.917374 21679 ts_manager.cc:194] Registered new tserver with Master: 24b875fef6ec46d0a67d2fa542756cf2 (127.18.39.67:41153)
I20260619 06:50:48.918193 21679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:58715
I20260619 06:50:48.929587 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:48.978612 22137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.963614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.18.39.126:34289)
0619 06:50:48.963784 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:48.963789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:48.964335 (+ 546us) server_negotiation.cc:407] Connection header received
0619 06:50:48.964496 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:48.964504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:48.964567 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:48.964657 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:48.966351 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.967101 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.968425 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.968874 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.968984 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.973392 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:48.973398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:48.973402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:48.973438 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:48.975932 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.976641 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.976647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.976651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.976823 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.977244 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.977272 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.977274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.977703 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:48.977905 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:48.978163 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 06:50:48.978379 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 06:50:48.979233 21679 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:32970
I20260619 06:50:48.981086 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21740
Jun 19 06:50:48 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:48.996681 22147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:48.982059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58896 (local address 127.18.39.65:38533)
0619 06:50:48.982624 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:48.982631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:48.982655 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:50:48.982714 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:48.982720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:48.982920 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:48.983042 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:48.984125 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.985103 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.986356 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:48.986913 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:48.987017 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:48.990867 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:48.990880 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:48.990895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:48.990931 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:48.993805 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.994552 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.994560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.994562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.994696 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:48.995081 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:48.995086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:48.995088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:48.995505 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:48.995766 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:48.996010 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 06:50:48.996308 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":272,"threads_started":1}
W20260619 06:50:48.998025 21863 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58896
W20260619 06:50:48.999123 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21740: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:48.999308 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21740
I20260619 06:50:49.015197 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21874
Jun 19 06:50:49 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:49.030594 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:49.016136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.18.39.66:36641)
0619 06:50:49.016736 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:49.016743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:49.016769 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:50:49.016834 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:49.016841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:49.017102 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:49.017230 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:49.018445 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:49.019480 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:49.020642 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:49.021190 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:49.021291 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:49.025418 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:49.025431 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:49.025453 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:49.025495 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:49.028112 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:49.028715 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:49.028722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:49.028724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:49.028844 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:49.029153 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:49.029156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:49.029158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:49.029602 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:49.029817 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:49.030097 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 06:50:49.030288 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":329,"threads_started":1}
W20260619 06:50:49.031939 21997 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42442
W20260619 06:50:49.033038 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21874: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:49.033175 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21874
I20260619 06:50:49.048233 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22007
Jun 19 06:50:49 dist-test-slave-1h5s krb5kdc[21634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851848, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:49.065088 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:49.049211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51992 (local address 127.18.39.67:41153)
0619 06:50:49.049831 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:49.049838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:49.049865 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:50:49.049921 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:49.049927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:49.050126 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:49.050340 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:49.051652 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:49.052688 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:49.054010 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:49.054610 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:49.054743 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:49.059126 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:49.059139 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:49.059155 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:49.059194 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:49.062197 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:49.062960 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:49.062967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:49.062969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:49.063099 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:49.063473 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:49.063484 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:49.063486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:49.063823 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:49.064140 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:49.064456 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 06:50:49.064732 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":338,"threads_started":1}
W20260619 06:50:49.066423 22130 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51992
W20260619 06:50:49.067725 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22007: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:49.067962 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22007
I20260619 06:50:49.083240 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 21647
I20260619 06:50:49.084444 21709 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:49.188427 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 21647
2026-06-19T06:50:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3653 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:49 dist-test-slave-1h5s krb5kdc[22153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:49 dist-test-slave-1h5s krb5kdc[22153](info): set up 2 sockets
Jun 19 06:50:49 dist-test-slave-1h5s krb5kdc[22153](info): commencing operation
krb5kdc: starting...
W20260619 06:50:50.277732 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.005s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:50 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:50.474380 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:43003
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:43003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:50.800146 22166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:50.800448 22166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:50.800601 22166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:50.807868 22166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:50.807981 22166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:50.808073 22166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:50.808177 22166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:50.808252 22166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:50.818264 22166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:43003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:43003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:50.819651 22166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:50.820997 22166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:50.832741 22172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:50.832710 22174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:50.833302 22171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:50.833873 22166 server_base.cc:1034] running on GCE node
I20260619 06:50:50.834661 22166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:50.836089 22166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:50.837319 22166 hybrid_clock.cc:648] HybridClock initialized: now 1781851850837261 us; error 48 us; skew 500 ppm
I20260619 06:50:50.837698 22166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:50 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851850, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:50.847476 22166 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:50.848981 22166 webserver.cc:460] Webserver started at http://127.18.39.126:46593/ using document root <none> and password file <none>
I20260619 06:50:50.849742 22166 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:50.849852 22166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:50.850116 22166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:50.853168 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "97363a245ba24a8986ccdfc3907b3ff3"
format_stamp: "Formatted at 2026-06-19 06:50:50 on dist-test-slave-1h5s"
server_key: "3d3321defa6a6d1faf2d8562ab8fc604"
server_key_iv: "be76ebb9ba763d139b436a697ca7e6bc"
server_key_version: "encryptionkey@0"
I20260619 06:50:50.853993 22166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "97363a245ba24a8986ccdfc3907b3ff3"
format_stamp: "Formatted at 2026-06-19 06:50:50 on dist-test-slave-1h5s"
server_key: "3d3321defa6a6d1faf2d8562ab8fc604"
server_key_iv: "be76ebb9ba763d139b436a697ca7e6bc"
server_key_version: "encryptionkey@0"
I20260619 06:50:50.859156 22166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:50:50.862809 22181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:50.864188 22166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:50:50.864420 22166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "97363a245ba24a8986ccdfc3907b3ff3"
format_stamp: "Formatted at 2026-06-19 06:50:50 on dist-test-slave-1h5s"
server_key: "3d3321defa6a6d1faf2d8562ab8fc604"
server_key_iv: "be76ebb9ba763d139b436a697ca7e6bc"
server_key_version: "encryptionkey@0"
I20260619 06:50:50.864609 22166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:50.883780 22166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:50.888320 22166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:50.888624 22166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:50.906457 22166 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:43003
I20260619 06:50:50.906462 22232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:43003 every 8 connection(s)
I20260619 06:50:50.907908 22166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:50.912185 22233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:50.913172 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22166
I20260619 06:50:50.913446 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:50.914065 18589 external_mini_cluster.cc:1428] Setting key 17190bf4d04047358507af4881a5ec2e
I20260619 06:50:50.926999 22233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3: Bootstrap starting.
Jun 19 06:50:50 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:50.930600 22233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:50.932080 22233 log.cc:824] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:50.935599 22233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3: No bootstrap required, opened a new log
I20260619 06:50:50.939184 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:50.916133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49816 (local address 127.18.39.126:43003)
0619 06:50:50.916977 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:50.916993 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:50.917043 (+ 50us) server_negotiation.cc:407] Connection header received
0619 06:50:50.918935 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:50.918968 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:50.919539 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:50.920016 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:50.921408 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:50.923442 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:50.924841 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:50.925994 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:50.926177 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:50.930405 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:50.930419 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:50.930442 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:50.930507 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:50.934338 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:50.935136 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:50.935144 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:50.935153 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:50.935368 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:50.935762 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:50.935767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:50.935770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:50.936524 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:50.936792 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:50.937172 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 06:50:50.937673 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":364,"threads_started":1}
I20260619 06:50:50.944384 22233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } }
I20260619 06:50:50.944754 22233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:50.944875 22233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97363a245ba24a8986ccdfc3907b3ff3, State: Initialized, Role: FOLLOWER
I20260619 06:50:50.945497 22233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } }
I20260619 06:50:50.945744 22233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:50.945888 22233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:50.946070 22233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:50.948231 22233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } }
I20260619 06:50:50.948760 22233 leader_election.cc:302] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97363a245ba24a8986ccdfc3907b3ff3; no voters:
I20260619 06:50:50.949508 22233 leader_election.cc:288] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:50.949615 22238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:50.950284 22238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [term 1 LEADER]: Becoming Leader. State: Replica: 97363a245ba24a8986ccdfc3907b3ff3, State: Running, Role: LEADER
I20260619 06:50:50.950892 22238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } }
I20260619 06:50:50.951833 22233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:50.953817 22240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97363a245ba24a8986ccdfc3907b3ff3. Latest consensus state: current_term: 1 leader_uuid: "97363a245ba24a8986ccdfc3907b3ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } } }
I20260619 06:50:50.954217 22240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:50.954634 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97363a245ba24a8986ccdfc3907b3ff3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97363a245ba24a8986ccdfc3907b3ff3" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 43003 } } }
I20260619 06:50:50.954907 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:50.957998 22245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:50.963387 22245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:50.972316 22245 catalog_manager.cc:1347] Generated new cluster ID: 82846c24449343069cd3a14596d5919b
I20260619 06:50:50.972532 22245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:50.989928 22245 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:50.991298 22245 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:51.000197 22245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97363a245ba24a8986ccdfc3907b3ff3: Generated new TSK 0
I20260619 06:50:51.001362 22245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:51.054826 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:51.388130 22259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:51.388449 22259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:51.388578 22259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:51.396297 22259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:51.396433 22259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:51.396597 22259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:51.406914 22259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:51.408268 22259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:51.409621 22259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:51.423102 22265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:51.423323 22267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:51.423254 22264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:51.424444 22259 server_base.cc:1034] running on GCE node
I20260619 06:50:51.425114 22259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:51.426204 22259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:51.427443 22259 hybrid_clock.cc:648] HybridClock initialized: now 1781851851427394 us; error 52 us; skew 500 ppm
I20260619 06:50:51.427769 22259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:51 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:51.430974 22259 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:51.432482 22259 webserver.cc:460] Webserver started at http://127.18.39.65:35977/ using document root <none> and password file <none>
I20260619 06:50:51.433288 22259 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:51.433466 22259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:51.433789 22259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:51.436700 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "c8d5dd4de01f4d47bda0381f05467b8f"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "21a19e852b9360333dc820a2c45b85df"
server_key_iv: "75463cb5220fbfc91525312dc9644a06"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.437594 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "c8d5dd4de01f4d47bda0381f05467b8f"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "21a19e852b9360333dc820a2c45b85df"
server_key_iv: "75463cb5220fbfc91525312dc9644a06"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.443132 22259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:50:51.446841 22274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:51.448304 22259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:50:51.448443 22259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "c8d5dd4de01f4d47bda0381f05467b8f"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "21a19e852b9360333dc820a2c45b85df"
server_key_iv: "75463cb5220fbfc91525312dc9644a06"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.448626 22259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:51.464162 22259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:51.468544 22259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:51.468847 22259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:51.470021 22259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:51.471405 22259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:51.471477 22259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:51.471650 22259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:51.471712 22259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:51.505023 22259 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:38875
I20260619 06:50:51.505101 22386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:38875 every 8 connection(s)
I20260619 06:50:51.506350 22259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:51.514963 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22259
I20260619 06:50:51.515195 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:51.515810 18589 external_mini_cluster.cc:1428] Setting key 0b8bb4af01b94a1917e20a88ee71aff5
Jun 19 06:50:51 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:51.526333 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:51.508798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42641 (local address 127.18.39.126:43003)
0619 06:50:51.509246 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:51.509250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:51.510296 (+ 1046us) server_negotiation.cc:407] Connection header received
0619 06:50:51.512101 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:51.512107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:51.512196 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:51.512315 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:51.513842 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:51.514653 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:51.516408 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:51.516964 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:51.517110 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:51.520554 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:51.520560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:51.520563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:51.520596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:51.523195 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:51.523958 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:51.523968 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:51.523971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:51.524086 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:51.524515 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:51.524520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:51.524523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:51.524794 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:51.524944 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:51.525736 (+ 792us) server_negotiation.cc:299] Negotiation successful
0619 06:50:51.525956 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":331,"thread_start_us":211,"threads_started":1}
I20260619 06:50:51.527371 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:51.509450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:43003 (local address 127.18.39.65:42641)
0619 06:50:51.510134 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:51.510170 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:51.511840 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:51.512553 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:51.512561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:51.513049 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:51.513677 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:51.513695 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:51.514844 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:51.514848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:51.516260 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:51.516275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:51.517195 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:51.517205 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:51.517511 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:51.518194 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:51.518217 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:51.520323 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:51.523359 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:51.523369 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:51.523387 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:51.523825 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:51.524194 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:51.524200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:51.524204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:51.524398 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:51.524923 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:51.524932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:51.525288 (+ 356us) client_negotiation.cc:769] Sending connection context
0619 06:50:51.525633 (+ 345us) client_negotiation.cc:240] Negotiation successful
0619 06:50:51.526221 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":273,"threads_started":1}
I20260619 06:50:51.529093 22387 heartbeater.cc:347] Connected to a master server at 127.18.39.126:43003
I20260619 06:50:51.529480 22387 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:51.530490 22387 heartbeater.cc:511] Master 127.18.39.126:43003 requested a full tablet report, sending...
I20260619 06:50:51.533218 22198 ts_manager.cc:194] Registered new tserver with Master: c8d5dd4de01f4d47bda0381f05467b8f (127.18.39.65:38875)
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:50:51.534932 22198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:42641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:51.560089 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:51.887204 22393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:51.887516 22393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:51.887614 22393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:51.894928 22393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:51.895061 22393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:51.895179 22393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:51.905174 22393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:51.906515 22393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:51.907912 22393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:51.921710 22399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:51.922026 22398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:51.922286 22401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:51.922622 22393 server_base.cc:1034] running on GCE node
I20260619 06:50:51.923295 22393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:51.924293 22393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:51.925518 22393 hybrid_clock.cc:648] HybridClock initialized: now 1781851851925469 us; error 40 us; skew 500 ppm
I20260619 06:50:51.925850 22393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:51 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:51.929368 22393 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:51.930817 22393 webserver.cc:460] Webserver started at http://127.18.39.66:36715/ using document root <none> and password file <none>
I20260619 06:50:51.931507 22393 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:51.931608 22393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:51.931876 22393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:51.934854 22393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "89e580108c174634acd50e50c6d25026"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "8771fdfa740c0bac896601c918b9401d"
server_key_iv: "04ea0194aa0e4a97fc522754a508b037"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.935636 22393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "89e580108c174634acd50e50c6d25026"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "8771fdfa740c0bac896601c918b9401d"
server_key_iv: "04ea0194aa0e4a97fc522754a508b037"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.941243 22393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:50:51.944905 22408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:51.946364 22393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:51.946508 22393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "89e580108c174634acd50e50c6d25026"
format_stamp: "Formatted at 2026-06-19 06:50:51 on dist-test-slave-1h5s"
server_key: "8771fdfa740c0bac896601c918b9401d"
server_key_iv: "04ea0194aa0e4a97fc522754a508b037"
server_key_version: "encryptionkey@0"
I20260619 06:50:51.946692 22393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:51.961488 22393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:51.966231 22393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:51.966475 22393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:51.967408 22393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:51.968870 22393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:51.968945 22393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:51.969031 22393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:51.969081 22393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:52.002267 22393 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:44097
I20260619 06:50:52.002372 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:44097 every 8 connection(s)
I20260619 06:50:52.003609 22393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:52.009640 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22393
I20260619 06:50:52.009840 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:52.010281 18589 external_mini_cluster.cc:1428] Setting key ad5bd7d05e262186a34c2be332936a37
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851851, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:52.025599 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.006360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:46359 (local address 127.18.39.126:43003)
0619 06:50:52.006552 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.006557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.007892 (+ 1335us) server_negotiation.cc:407] Connection header received
0619 06:50:52.010245 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.010252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.010362 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.010503 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.012492 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.013288 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.015062 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.015527 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.015679 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.019793 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.019799 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.019802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.019835 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.022492 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.023279 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.023287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.023295 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.023422 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.023844 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.023851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.023854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.024128 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.024283 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.025002 (+ 719us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.025225 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:50:52.026623 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.006865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:43003 (local address 127.18.39.66:46359)
0619 06:50:52.007693 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:52.007735 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:52.009916 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:52.010821 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:52.010835 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:52.011523 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:52.012288 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:52.012318 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.013523 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.013534 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:52.014897 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:52.014911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.015783 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.015791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:52.016051 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.016824 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:52.016851 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:52.019526 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:52.022654 (+ 3128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:52.022665 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:52.022685 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:52.023132 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:52.023535 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:52.023540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:52.023544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:52.023737 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:52.024266 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:52.024275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:52.024639 (+ 364us) client_negotiation.cc:769] Sending connection context
0619 06:50:52.024906 (+ 267us) client_negotiation.cc:240] Negotiation successful
0619 06:50:52.025501 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":360,"threads_started":1}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:52.028215 22521 heartbeater.cc:347] Connected to a master server at 127.18.39.126:43003
I20260619 06:50:52.028570 22521 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:52.029413 22521 heartbeater.cc:511] Master 127.18.39.126:43003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:50:52.031096 22198 ts_manager.cc:194] Registered new tserver with Master: 89e580108c174634acd50e50c6d25026 (127.18.39.66:44097)
I20260619 06:50:52.031989 22198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:46359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:52.056836 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:52.383185 22526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:52.383527 22526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:52.383694 22526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:52.391294 22526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:52.391419 22526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:52.391606 22526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:52.402221 22526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:43003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:52.403549 22526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:52.404862 22526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:52.418072 22531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:52.418162 22534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:52.418188 22532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:52.419054 22526 server_base.cc:1034] running on GCE node
I20260619 06:50:52.419740 22526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:52.420684 22526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:52.421908 22526 hybrid_clock.cc:648] HybridClock initialized: now 1781851852421860 us; error 59 us; skew 500 ppm
I20260619 06:50:52.422216 22526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:52.425473 22526 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:52.426863 22526 webserver.cc:460] Webserver started at http://127.18.39.67:37873/ using document root <none> and password file <none>
I20260619 06:50:52.427561 22526 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:52.427706 22526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:52.428015 22526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:52.431089 22526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "56428652872c470496cb0632925a7e62"
format_stamp: "Formatted at 2026-06-19 06:50:52 on dist-test-slave-1h5s"
server_key: "6ab3859f71c0c1f31073532f638135b0"
server_key_iv: "3a35e9d32f991ec0d40964fcdf54059a"
server_key_version: "encryptionkey@0"
I20260619 06:50:52.431895 22526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "56428652872c470496cb0632925a7e62"
format_stamp: "Formatted at 2026-06-19 06:50:52 on dist-test-slave-1h5s"
server_key: "6ab3859f71c0c1f31073532f638135b0"
server_key_iv: "3a35e9d32f991ec0d40964fcdf54059a"
server_key_version: "encryptionkey@0"
I20260619 06:50:52.437004 22526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:50:52.440606 22541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:52.442035 22526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:52.442220 22526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "56428652872c470496cb0632925a7e62"
format_stamp: "Formatted at 2026-06-19 06:50:52 on dist-test-slave-1h5s"
server_key: "6ab3859f71c0c1f31073532f638135b0"
server_key_iv: "3a35e9d32f991ec0d40964fcdf54059a"
server_key_version: "encryptionkey@0"
I20260619 06:50:52.442440 22526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:52.461680 22526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:52.466148 22526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:52.466435 22526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:52.467371 22526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:52.468843 22526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:52.468946 22526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:52.469076 22526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:52.469146 22526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:52.501055 22526 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:37583
I20260619 06:50:52.501097 22653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:37583 every 8 connection(s)
I20260619 06:50:52.502468 22526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:52.506911 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22526
I20260619 06:50:52.507107 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:52.507531 18589 external_mini_cluster.cc:1428] Setting key 4099afb55beaebd93a59790549ab1f9a
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:52.523415 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.505244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:53793 (local address 127.18.39.126:43003)
0619 06:50:52.505488 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.505495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.506788 (+ 1293us) server_negotiation.cc:407] Connection header received
0619 06:50:52.509327 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.509358 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.509484 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.509591 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.511402 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.512142 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.513704 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.514104 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.514198 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.517452 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.517457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.517460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.517491 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.520123 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.520861 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.520867 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.520870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.521062 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.521651 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.521658 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.521666 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.521959 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.522117 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.522833 (+ 716us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.523058 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:50:52.524412 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.505784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:43003 (local address 127.18.39.67:53793)
0619 06:50:52.506614 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:52.506669 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:52.509007 (+ 2338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:52.509867 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:52.509879 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:52.510600 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:52.511241 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:52.511259 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.512269 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.512272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:52.513570 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:52.513581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.514294 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.514299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:52.514496 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.515107 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:52.515128 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:52.517204 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:52.520261 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:52.520268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:52.520283 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:52.520720 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:52.521208 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:52.521213 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:52.521216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:52.521496 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:52.522091 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:52.522100 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:52.522467 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 06:50:52.522742 (+ 275us) client_negotiation.cc:240] Negotiation successful
0619 06:50:52.523294 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":303,"threads_started":1}
I20260619 06:50:52.525781 22654 heartbeater.cc:347] Connected to a master server at 127.18.39.126:43003
I20260619 06:50:52.526093 22654 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:52.526872 22654 heartbeater.cc:511] Master 127.18.39.126:43003 requested a full tablet report, sending...
I20260619 06:50:52.528352 22198 ts_manager.cc:194] Registered new tserver with Master: 56428652872c470496cb0632925a7e62 (127.18.39.67:37583)
I20260619 06:50:52.529188 22198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:53793
I20260619 06:50:52.535238 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:52.538218 22387 heartbeater.cc:503] Master 127.18.39.126:43003 was elected leader, sending a full tablet report...
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:52.584936 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.569395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.18.39.65:38875)
0619 06:50:52.570046 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.570056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.570090 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:50:52.570168 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.570181 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.570401 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.570586 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.572047 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.573071 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.574429 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.575004 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.575143 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.579111 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.579125 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.579141 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.579178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.581959 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.582715 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.582723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.582725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.582851 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.583285 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.583290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.583292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.583651 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.583961 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.584270 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.584562 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":306,"threads_started":1}
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:52.602898 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.589194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.18.39.126:43003)
0619 06:50:52.589505 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.589512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.589639 (+ 127us) server_negotiation.cc:407] Connection header received
0619 06:50:52.589843 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.589849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.589920 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.590030 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.591308 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.592058 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.593329 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.593789 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.593935 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.597800 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.597805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.597808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.597841 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.600161 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.600855 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.600862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.600866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.601019 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.601420 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.601426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.601429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.601757 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.602047 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.602356 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.602574 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260619 06:50:52.603538 22198 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49820
I20260619 06:50:52.604593 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22259
I20260619 06:50:52.618363 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.605671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.18.39.65:38875)
0619 06:50:52.605881 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.605888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.606041 (+ 153us) server_negotiation.cc:407] Connection header received
0619 06:50:52.606245 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.606249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.606317 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.606422 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.607647 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.608329 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.609613 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.610046 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.610186 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.613174 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.613179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.613183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.613217 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.615973 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.616651 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.616656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.616658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.616818 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.617148 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.617151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.617153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.617507 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.617684 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.617954 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.618119 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:50:52.619009 22382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:52.703366 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22259
I20260619 06:50:52.719558 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22393
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:52.735633 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.720485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.18.39.66:44097)
0619 06:50:52.721077 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.721085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.721111 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:50:52.721171 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.721178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.721475 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.721610 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.722748 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.723746 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.724976 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.725640 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.725740 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.730019 (+ 4279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.730034 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.730050 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.730087 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.733078 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.733781 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.733788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.733790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.733926 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.734236 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.734239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.734241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.734587 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.734808 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.735034 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.735219 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":314,"threads_started":1}
I20260619 06:50:52.736874 22516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:52.819123 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22393
I20260619 06:50:52.834600 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22526
Jun 19 06:50:52 dist-test-slave-1h5s krb5kdc[22153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:52.850848 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:52.835552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57002 (local address 127.18.39.67:37583)
0619 06:50:52.836110 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:52.836116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:52.836144 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:50:52.836202 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:52.836209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:52.836421 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:52.836560 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:52.837887 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.838915 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.840165 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:52.840725 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:52.840819 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:52.845195 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:52.845210 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:52.845225 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:52.845266 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:52.848219 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.848885 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.848891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.848894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.849012 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:52.849385 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:52.849389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:52.849391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:52.849740 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:52.849976 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:52.850233 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 06:50:52.850437 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":261,"threads_started":1}
I20260619 06:50:52.852272 22649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:52.935251 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22526
I20260619 06:50:52.950709 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22166
I20260619 06:50:52.952061 22228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:53.055079 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22166
2026-06-19T06:50:53Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3867 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:53 dist-test-slave-1h5s krb5kdc[22673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:53 dist-test-slave-1h5s krb5kdc[22673](info): set up 2 sockets
Jun 19 06:50:53 dist-test-slave-1h5s krb5kdc[22673](info): commencing operation
krb5kdc: starting...
W20260619 06:50:54.165782 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:50:54 dist-test-slave-1h5s krb5kdc[22673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:50:54.388967 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39941
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:39941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:54.715706 22686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:54.716064 22686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:54.716243 22686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:54.723518 22686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:54.723702 22686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:54.723802 22686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:54.723894 22686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:54.723986 22686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:54.734017 22686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:39941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:54.735416 22686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:54.736824 22686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:54.749100 22694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:54.749220 22692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:54.749225 22691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:54.750563 22686 server_base.cc:1034] running on GCE node
I20260619 06:50:54.751464 22686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:54.752962 22686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:54.754196 22686 hybrid_clock.cc:648] HybridClock initialized: now 1781851854754170 us; error 51 us; skew 500 ppm
I20260619 06:50:54.754561 22686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:54 dist-test-slave-1h5s krb5kdc[22673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:54.757814 22686 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:50:54.759042 22686 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:54.759181 22686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:54.759516 22686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:54.762683 22686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "ce18cf9951854bb78c7933654d0ac07b"
format_stamp: "Formatted at 2026-06-19 06:50:54 on dist-test-slave-1h5s"
server_key: "27a16278570e8c753b0bcf50934ceb70"
server_key_iv: "84cffa6f093d5486801d6fa19470a8ee"
server_key_version: "encryptionkey@0"
I20260619 06:50:54.763520 22686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "ce18cf9951854bb78c7933654d0ac07b"
format_stamp: "Formatted at 2026-06-19 06:50:54 on dist-test-slave-1h5s"
server_key: "27a16278570e8c753b0bcf50934ceb70"
server_key_iv: "84cffa6f093d5486801d6fa19470a8ee"
server_key_version: "encryptionkey@0"
I20260619 06:50:54.768836 22686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 06:50:54.772642 22700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:54.774072 22686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:54.774267 22686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "ce18cf9951854bb78c7933654d0ac07b"
format_stamp: "Formatted at 2026-06-19 06:50:54 on dist-test-slave-1h5s"
server_key: "27a16278570e8c753b0bcf50934ceb70"
server_key_iv: "84cffa6f093d5486801d6fa19470a8ee"
server_key_version: "encryptionkey@0"
I20260619 06:50:54.774499 22686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:54.796531 22686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:54.801047 22686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:54.801326 22686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:54.820051 22686 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:39941
I20260619 06:50:54.820053 22751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:39941 every 8 connection(s)
I20260619 06:50:54.821020 22686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:54.824780 22752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:54.827839 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22686
I20260619 06:50:54.828070 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:54.828692 18589 external_mini_cluster.cc:1428] Setting key 0d8b48527d24a65f1121e57ab966c15a
I20260619 06:50:54.837723 22752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: Bootstrap starting.
I20260619 06:50:54.841778 22752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:54.843240 22752 log.cc:824] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:50:54 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:54.846462 22752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: No bootstrap required, opened a new log
I20260619 06:50:54.852439 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:54.830711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.18.39.126:39941)
0619 06:50:54.831568 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:54.831579 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:54.831623 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:50:54.833275 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:54.833299 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:54.833805 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:54.834154 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:54.836099 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:54.837471 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:54.839235 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:54.839921 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:54.840046 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:54.844953 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:54.844963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:54.844979 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:54.845020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:54.848125 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:54.848887 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:54.848895 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:54.848902 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:54.849069 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:54.849536 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:54.849542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:54.849548 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:54.850099 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:54.850409 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:54.850890 (+ 481us) server_negotiation.cc:299] Negotiation successful
0619 06:50:54.851325 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":395,"threads_started":1}
I20260619 06:50:54.854843 22752 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } }
I20260619 06:50:54.855304 22752 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:54.855470 22752 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce18cf9951854bb78c7933654d0ac07b, State: Initialized, Role: FOLLOWER
I20260619 06:50:54.856091 22752 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } }
I20260619 06:50:54.856294 22752 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:54.856389 22752 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:54.856506 22752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:54.859133 22752 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } }
I20260619 06:50:54.859593 22752 leader_election.cc:302] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce18cf9951854bb78c7933654d0ac07b; no voters:
I20260619 06:50:54.860090 22752 leader_election.cc:288] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:54.860241 22757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:54.860925 22757 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [term 1 LEADER]: Becoming Leader. State: Replica: ce18cf9951854bb78c7933654d0ac07b, State: Running, Role: LEADER
I20260619 06:50:54.861533 22757 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } }
I20260619 06:50:54.862469 22752 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:54.864490 22759 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce18cf9951854bb78c7933654d0ac07b. Latest consensus state: current_term: 1 leader_uuid: "ce18cf9951854bb78c7933654d0ac07b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } } }
I20260619 06:50:54.864681 22758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce18cf9951854bb78c7933654d0ac07b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce18cf9951854bb78c7933654d0ac07b" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39941 } } }
I20260619 06:50:54.864946 22759 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:54.865020 22758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:54.866564 22765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:54.871403 22765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:54.880880 22765 catalog_manager.cc:1347] Generated new cluster ID: ba4d83fcd1d444708902b55a9bdf51e6
I20260619 06:50:54.880997 22765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:54.898557 22765 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:54.900194 22765 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:54.908124 22765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: Generated new TSK 0
I20260619 06:50:54.909256 22765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:50:54.973598 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:55.318358 22778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:55.318670 22778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:55.318846 22778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:55.326450 22778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:55.326571 22778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:55.326761 22778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:55.336867 22778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:55.338291 22778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:55.339675 22778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:55.352967 22783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:55.353061 22784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:55.353027 22786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:55.354321 22778 server_base.cc:1034] running on GCE node
I20260619 06:50:55.354997 22778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:55.356021 22778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:55.357285 22778 hybrid_clock.cc:648] HybridClock initialized: now 1781851855357215 us; error 39 us; skew 500 ppm
I20260619 06:50:55.357677 22778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:55 dist-test-slave-1h5s krb5kdc[22673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:55.360828 22778 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:50:55.362061 22778 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:55.362195 22778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:55.362524 22778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:55.365684 22778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "915135f0c7b3459eb7b654fe6e973f8f"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "2ab23908811beb34cb4c7fcf5583aa37"
server_key_iv: "3e981a5a126e7e97aab6d267c9a0865a"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.366647 22778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "915135f0c7b3459eb7b654fe6e973f8f"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "2ab23908811beb34cb4c7fcf5583aa37"
server_key_iv: "3e981a5a126e7e97aab6d267c9a0865a"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.372037 22778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260619 06:50:55.375738 22792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.377245 22778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:55.377519 22778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "915135f0c7b3459eb7b654fe6e973f8f"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "2ab23908811beb34cb4c7fcf5583aa37"
server_key_iv: "3e981a5a126e7e97aab6d267c9a0865a"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.377826 22778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:55.400149 22778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:55.404330 22778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:55.404618 22778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:55.405671 22778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:55.407202 22778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:55.407346 22778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.407490 22778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:55.407586 22778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.441155 22778 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:33343
I20260619 06:50:55.441208 22904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:33343 every 8 connection(s)
I20260619 06:50:55.442148 22778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:55.447019 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22778
I20260619 06:50:55.447288 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:55.447954 18589 external_mini_cluster.cc:1428] Setting key 00981322ab31c11ee16655e57fa9801d
Jun 19 06:50:55 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:55.466979 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:55.444951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46643 (local address 127.18.39.126:39941)
0619 06:50:55.445514 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:55.445519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:55.446652 (+ 1133us) server_negotiation.cc:407] Connection header received
0619 06:50:55.449135 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:55.449141 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:55.449268 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:55.449431 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:55.451328 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.452082 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.453932 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.454350 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.454497 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:55.458964 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:55.458974 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:55.458982 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:55.459033 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:55.462844 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:55.464044 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:55.464054 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:55.464060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:55.464224 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:55.464836 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:55.464845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:55.464851 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:55.465278 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:55.465501 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:55.466259 (+ 758us) server_negotiation.cc:299] Negotiation successful
0619 06:50:55.466536 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":446,"thread_start_us":256,"threads_started":1}
I20260619 06:50:55.467790 22908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:55.445618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39941 (local address 127.18.39.65:46643)
0619 06:50:55.446449 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:55.446497 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:55.448821 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:55.449717 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:55.449727 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:55.450474 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:55.451120 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:55.451139 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.452251 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.452261 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:55.453748 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:55.453762 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.454602 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.454611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:55.454866 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:55.455705 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:55.455741 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:55.458669 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:55.463061 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:55.463071 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:55.463087 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:55.463868 (+ 781us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:55.464349 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:55.464353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:55.464355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:55.464686 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:55.465468 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:55.465475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:55.465840 (+ 365us) client_negotiation.cc:769] Sending connection context
0619 06:50:55.466079 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 06:50:55.466521 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":346,"threads_started":1}
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:50:55.469805 22905 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39941
I20260619 06:50:55.470230 22905 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:55.471243 22905 heartbeater.cc:511] Master 127.18.39.126:39941 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:50:55.474102 22717 ts_manager.cc:194] Registered new tserver with Master: 915135f0c7b3459eb7b654fe6e973f8f (127.18.39.65:33343)
I20260619 06:50:55.475756 22717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:46643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:50:55.497216 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:55.831544 22911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:55.831812 22911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:55.831987 22911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:55.839319 22911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:55.839440 22911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:55.839639 22911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:55.849773 22911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:55.851246 22911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:55.852607 22911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:55.867049 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:55.867134 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:55.867089 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:55.870944 22911 server_base.cc:1034] running on GCE node
I20260619 06:50:55.871629 22911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:55.872650 22911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:55.873943 22911 hybrid_clock.cc:648] HybridClock initialized: now 1781851855873895 us; error 54 us; skew 500 ppm
I20260619 06:50:55.874333 22911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:55 dist-test-slave-1h5s krb5kdc[22673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:55.880918 22911 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:50:55.882424 22911 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:55.882612 22911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:55.883015 22911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:55.886252 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "7d561ae0a12c473d8d2b66d42690534a"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "fa0695c8ecb42896d43075982e1185de"
server_key_iv: "a6d5d8632350ba934b0c329d54896645"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.887108 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "7d561ae0a12c473d8d2b66d42690534a"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "fa0695c8ecb42896d43075982e1185de"
server_key_iv: "a6d5d8632350ba934b0c329d54896645"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.892616 22911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:55.896354 22925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.897845 22911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:50:55.898054 22911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "7d561ae0a12c473d8d2b66d42690534a"
format_stamp: "Formatted at 2026-06-19 06:50:55 on dist-test-slave-1h5s"
server_key: "fa0695c8ecb42896d43075982e1185de"
server_key_iv: "a6d5d8632350ba934b0c329d54896645"
server_key_version: "encryptionkey@0"
I20260619 06:50:55.898293 22911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:55.911942 22911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:55.916177 22911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:55.916468 22911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:55.917557 22911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:55.918947 22911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:55.919080 22911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.919216 22911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:55.919304 22911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:55.953230 22911 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:38291
I20260619 06:50:55.953392 23037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:38291 every 8 connection(s)
I20260619 06:50:55.954154 22911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:55.957176 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 22911
I20260619 06:50:55.957388 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:55.958053 18589 external_mini_cluster.cc:1428] Setting key d02cbfe2c69e02bcfe1a5fb2043baff4
Jun 19 06:50:55 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:55.978618 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:55.956729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:45075 (local address 127.18.39.126:39941)
0619 06:50:55.956990 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:55.956998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:55.958378 (+ 1380us) server_negotiation.cc:407] Connection header received
0619 06:50:55.960677 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:55.960682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:55.960753 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:55.960856 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:55.963159 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.964065 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.966658 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.967503 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.967684 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:55.972125 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:55.972130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:55.972133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:55.972166 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:55.974816 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:55.975946 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:55.975951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:55.975953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:55.976077 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:55.976651 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:55.976654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:55.976656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:55.976906 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:55.977015 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:55.978104 (+ 1089us) server_negotiation.cc:299] Negotiation successful
0619 06:50:55.978303 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:50:55.980393 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:55.957233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39941 (local address 127.18.39.66:45075)
0619 06:50:55.958177 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:55.958237 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:55.960343 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:55.961160 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:55.961174 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:55.962065 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:55.962944 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:55.962985 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.964233 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.964241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:55.966480 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:55.966494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:55.967729 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:55.967733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:55.967927 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:55.968734 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:55.968763 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:55.971851 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:55.975012 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:55.975026 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:55.975052 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:55.975806 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:55.976198 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:55.976206 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:55.976213 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:55.976526 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:55.977044 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:55.977056 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:55.977619 (+ 563us) client_negotiation.cc:769] Sending connection context
0619 06:50:55.978003 (+ 384us) client_negotiation.cc:240] Negotiation successful
0619 06:50:55.978690 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":310,"threads_started":1}
I20260619 06:50:55.982103 23038 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39941
I20260619 06:50:55.982510 23038 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:50:55.983433 23038 heartbeater.cc:511] Master 127.18.39.126:39941 requested a full tablet report, sending...
I20260619 06:50:55.985088 22717 ts_manager.cc:194] Registered new tserver with Master: 7d561ae0a12c473d8d2b66d42690534a (127.18.39.66:38291)
I20260619 06:50:55.985999 22717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:45075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:50:56.007093 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:56.330739 23043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:56.331110 23043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:56.331302 23043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:56.338711 23043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:56.338887 23043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:56.339076 23043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:56.349086 23043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:56.350508 23043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:56.351835 23043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:56.367451 23048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:56.367430 23049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:56.367360 23051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:56.368544 23043 server_base.cc:1034] running on GCE node
I20260619 06:50:56.369215 23043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:56.370337 23043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:56.371541 23043 hybrid_clock.cc:648] HybridClock initialized: now 1781851856371497 us; error 45 us; skew 500 ppm
I20260619 06:50:56.371886 23043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:56 dist-test-slave-1h5s krb5kdc[22673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:56.375439 23043 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:50:56.376725 23043 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:56.376821 23043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:56.377092 23043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:56.380342 23043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "1837c11167a240e1a0bb6447c49c2fa9"
format_stamp: "Formatted at 2026-06-19 06:50:56 on dist-test-slave-1h5s"
server_key: "1c38092beba844f7c4036278729ae6e7"
server_key_iv: "a36c58812c8c68b43ce70c5625521942"
server_key_version: "encryptionkey@0"
I20260619 06:50:56.381167 23043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "1837c11167a240e1a0bb6447c49c2fa9"
format_stamp: "Formatted at 2026-06-19 06:50:56 on dist-test-slave-1h5s"
server_key: "1c38092beba844f7c4036278729ae6e7"
server_key_iv: "a36c58812c8c68b43ce70c5625521942"
server_key_version: "encryptionkey@0"
I20260619 06:50:56.386808 23043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260619 06:50:56.390398 23057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:56.391929 23043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:56.392134 23043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "1837c11167a240e1a0bb6447c49c2fa9"
format_stamp: "Formatted at 2026-06-19 06:50:56 on dist-test-slave-1h5s"
server_key: "1c38092beba844f7c4036278729ae6e7"
server_key_iv: "a36c58812c8c68b43ce70c5625521942"
server_key_version: "encryptionkey@0"
I20260619 06:50:56.392387 23043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:56.407788 23043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:56.412385 23043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:56.412680 23043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:56.413724 23043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:56.415104 23043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:56.415230 23043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:56.415372 23043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:56.415462 23043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:56.449923 23043 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44897
I20260619 06:50:56.449968 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44897 every 8 connection(s)
I20260619 06:50:56.450780 23043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:56.455788 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23043
I20260619 06:50:56.455996 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:56.456503 18589 external_mini_cluster.cc:1428] Setting key 36122301c1826eddee29485258b0cccd
Jun 19 06:50:56 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:50:56.472482 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.453234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34193 (local address 127.18.39.126:39941)
0619 06:50:56.453466 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.453471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.454633 (+ 1162us) server_negotiation.cc:407] Connection header received
0619 06:50:56.457063 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.457069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.457178 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.457305 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:56.459343 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.460225 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.461861 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.462305 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.462439 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.466009 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:56.466014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:56.466018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:56.466049 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:56.468903 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:56.469789 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:56.469793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:56.469795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:56.469979 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:56.470482 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:56.470487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:56.470491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:56.470809 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:56.470922 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.471897 (+ 975us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.472120 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260619 06:50:56.473547 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.453784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39941 (local address 127.18.39.67:34193)
0619 06:50:56.454466 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.454507 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.456695 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.457615 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.457624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.458445 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:56.459155 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.459172 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.460383 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.460389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.461716 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.461727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.462530 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.462538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.462821 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.463505 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:56.463532 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:56.465801 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:56.469049 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:56.469057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:56.469072 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:56.469642 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:56.470098 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:56.470103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:56.470106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:56.470343 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:56.470939 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:56.470947 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:56.471432 (+ 485us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.471767 (+ 335us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.472356 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":278,"threads_started":1}
I20260619 06:50:56.474874 23170 heartbeater.cc:347] Connected to a master server at 127.18.39.126:39941
I20260619 06:50:56.475178 23170 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:56.475853 23170 heartbeater.cc:511] Master 127.18.39.126:39941 requested a full tablet report, sending...
I20260619 06:50:56.477198 22717 ts_manager.cc:194] Registered new tserver with Master: 1837c11167a240e1a0bb6447c49c2fa9 (127.18.39.67:44897)
I20260619 06:50:56.478082 22717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:34193
I20260619 06:50:56.480160 22905 heartbeater.cc:503] Master 127.18.39.126:39941 was elected leader, sending a full tablet report...
I20260619 06:50:56.484721 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:56.502122 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.489394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.18.39.126:39941)
0619 06:50:56.489563 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.489568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.490099 (+ 531us) server_negotiation.cc:407] Connection header received
0619 06:50:56.490227 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.490231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.490307 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.490401 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:56.491857 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.492542 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.494087 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.494550 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.494653 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.497159 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:56.497164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:56.497168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:56.497201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:56.499648 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:56.500339 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:56.500343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:56.500346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:56.500476 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:56.500781 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:56.500785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:56.500786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:56.501093 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:56.501307 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.501706 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.501896 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:50:56.506434 22717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:56.513757 22717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:56.548991 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.527574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.18.39.66:38291)
0619 06:50:56.537914 (+ 10340us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.537928 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.540328 (+ 2400us) server_negotiation.cc:407] Connection header received
0619 06:50:56.540470 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.540484 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.540857 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.541119 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:56.542434 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.544006 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.546823 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.547783 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.547939 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.548142 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.548485 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.548656 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10084,"thread_start_us":9944,"threads_started":1}
I20260619 06:50:56.549046 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.527046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:38291 (local address 127.0.0.1:38556)
0619 06:50:56.539935 (+ 12889us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.539964 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.540145 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.541736 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.541746 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.541779 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:56.542209 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.542222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.544146 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.544151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.546620 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.546639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.547998 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.548004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.548209 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.548265 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.548387 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.548572 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12711,"mutex_wait_us":30,"thread_start_us":16425,"threads_started":1}
I20260619 06:50:56.551131 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.525722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:33343 (local address 127.0.0.1:38118)
0619 06:50:56.526686 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.526728 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.526888 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.543358 (+ 16470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.543362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.543388 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:56.543779 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.543790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.545695 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.545703 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.547635 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.547648 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.550293 (+ 2645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.550300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.550473 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.550503 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.550595 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.550745 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":1,"thread_start_us":493,"threads_started":1}
I20260619 06:50:56.551440 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.525981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.18.39.65:33343)
0619 06:50:56.542713 (+ 16732us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.542721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.542765 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:50:56.542880 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.542888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.543132 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.543985 (+ 853us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:56.544202 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.545574 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.547801 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.550087 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.550231 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.550457 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.550936 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.551107 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16565,"thread_start_us":17619,"threads_started":1}
I20260619 06:50:56.551573 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.526388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44897 (local address 127.0.0.1:38116)
0619 06:50:56.543323 (+ 16935us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.543343 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.543730 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.544446 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.544450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.544479 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:56.544902 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.544913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.546821 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.546826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.548853 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.548865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.550719 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.550725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.550921 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.550953 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.551037 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.551160 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16796,"mutex_wait_us":692,"thread_start_us":14269,"threads_started":1}
I20260619 06:50:56.552330 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.526825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.18.39.67:44897)
0619 06:50:56.542056 (+ 15231us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.542067 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.543443 (+ 1376us) server_negotiation.cc:407] Connection header received
0619 06:50:56.543992 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.543998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.544226 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.544443 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:56.545116 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.546711 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.549046 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.550484 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.550613 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.550835 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.551837 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.551988 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15051,"thread_start_us":15245,"threads_started":1}
I20260619 06:50:56.553551 22973 tablet_service.cc:1467] Processing CreateTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 (DEFAULT_TABLE table=test-table [id=0df2bf3f88254d5b920a14e3865d1982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:56.555049 22973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86253a86fd9d4aa98e8296bd939f4ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:56.555001 23105 tablet_service.cc:1467] Processing CreateTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 (DEFAULT_TABLE table=test-table [id=0df2bf3f88254d5b920a14e3865d1982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:56.555001 22840 tablet_service.cc:1467] Processing CreateTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 (DEFAULT_TABLE table=test-table [id=0df2bf3f88254d5b920a14e3865d1982]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:56.556123 23105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86253a86fd9d4aa98e8296bd939f4ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:56.556289 22840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86253a86fd9d4aa98e8296bd939f4ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:56.567771 23188 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Bootstrap starting.
I20260619 06:50:56.569808 23189 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Bootstrap starting.
I20260619 06:50:56.571488 23190 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Bootstrap starting.
I20260619 06:50:56.571741 23188 tablet_bootstrap.cc:654] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:56.573065 23188 log.cc:824] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:56.573238 23189 tablet_bootstrap.cc:654] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:56.574347 23189 log.cc:824] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:56.574611 23190 tablet_bootstrap.cc:654] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:56.575820 23190 log.cc:824] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:56.576490 23188 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: No bootstrap required, opened a new log
I20260619 06:50:56.576776 23188 ts_tablet_manager.cc:1397] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260619 06:50:56.577827 23189 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: No bootstrap required, opened a new log
I20260619 06:50:56.578145 23189 ts_tablet_manager.cc:1397] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260619 06:50:56.583904 23190 tablet_bootstrap.cc:492] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: No bootstrap required, opened a new log
I20260619 06:50:56.584179 23190 ts_tablet_manager.cc:1397] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260619 06:50:56.584615 23188 raft_consensus.cc:348] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.584981 23188 raft_consensus.cc:374] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:56.585071 23188 raft_consensus.cc:729] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1837c11167a240e1a0bb6447c49c2fa9, State: Initialized, Role: FOLLOWER
I20260619 06:50:56.585708 23188 consensus_queue.cc:260] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.589206 23189 raft_consensus.cc:348] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.589686 23189 raft_consensus.cc:374] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:56.589775 23189 raft_consensus.cc:729] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d561ae0a12c473d8d2b66d42690534a, State: Initialized, Role: FOLLOWER
I20260619 06:50:56.590456 23189 consensus_queue.cc:260] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.591722 23038 heartbeater.cc:503] Master 127.18.39.126:39941 was elected leader, sending a full tablet report...
I20260619 06:50:56.593717 23189 ts_tablet_manager.cc:1428] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260619 06:50:56.594630 23170 heartbeater.cc:503] Master 127.18.39.126:39941 was elected leader, sending a full tablet report...
I20260619 06:50:56.594502 23190 raft_consensus.cc:348] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.594950 23190 raft_consensus.cc:374] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:56.595041 23190 raft_consensus.cc:729] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 915135f0c7b3459eb7b654fe6e973f8f, State: Initialized, Role: FOLLOWER
I20260619 06:50:56.595212 23188 ts_tablet_manager.cc:1428] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260619 06:50:56.595721 23190 consensus_queue.cc:260] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.597528 23190 ts_tablet_manager.cc:1428] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
W20260619 06:50:56.694603 22906 tablet.cc:2369] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:56.702416 23171 tablet.cc:2369] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:50:56.706054 23039 tablet.cc:2369] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:56.985708 23194 raft_consensus.cc:482] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:56.986039 23194 raft_consensus.cc:504] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.987876 23194 leader_election.cc:288] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 915135f0c7b3459eb7b654fe6e973f8f (127.18.39.65:33343), 7d561ae0a12c473d8d2b66d42690534a (127.18.39.66:38291)
I20260619 06:50:56.995551 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.988393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:32939 (local address 127.18.39.65:33343)
0619 06:50:56.988610 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.988619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.988789 (+ 170us) server_negotiation.cc:407] Connection header received
0619 06:50:56.989017 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.989026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.989135 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.989239 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:56.989866 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.990901 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.993431 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.994627 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.994831 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.994922 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.995134 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.995287 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:50:56.995801 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.988759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:38291 (local address 127.18.39.67:41765)
0619 06:50:56.989384 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.989406 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.989521 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.989817 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.989821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.989851 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:56.990144 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.990151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.991385 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.991389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.993449 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.993459 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.995048 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.995056 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.995231 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.995258 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.995342 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.995445 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":2,"thread_start_us":310,"threads_started":1}
I20260619 06:50:56.995899 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.988871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:41765 (local address 127.18.39.66:38291)
0619 06:50:56.989079 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.989086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:56.989479 (+ 393us) server_negotiation.cc:407] Connection header received
0619 06:50:56.989642 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:56.989645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:56.989703 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:56.989766 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:56.990295 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.991249 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.993580 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.994839 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.994988 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.995033 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:56.995377 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 06:50:56.995495 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:50:56.996053 23196 raft_consensus.cc:482] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:56.996282 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.988088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:33343 (local address 127.18.39.67:32939)
0619 06:50:56.988648 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:56.988676 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:56.988846 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:56.989281 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:56.989286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:56.989314 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:56.989695 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.989705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.991039 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.991043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.993214 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:56.993228 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:56.994770 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:56.994774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:56.994927 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:56.994973 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 06:50:56.995082 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 06:50:56.995233 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":238,"threads_started":1}
I20260619 06:50:56.996606 23196 raft_consensus.cc:504] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:56.996879 22993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "1837c11167a240e1a0bb6447c49c2fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d561ae0a12c473d8d2b66d42690534a" is_pre_election: true
I20260619 06:50:56.997431 22993 raft_consensus.cc:2454] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837c11167a240e1a0bb6447c49c2fa9 in term 0.
I20260619 06:50:56.997320 22860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "1837c11167a240e1a0bb6447c49c2fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915135f0c7b3459eb7b654fe6e973f8f" is_pre_election: true
I20260619 06:50:56.997838 22860 raft_consensus.cc:2454] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1837c11167a240e1a0bb6447c49c2fa9 in term 0.
I20260619 06:50:56.998137 23061 leader_election.cc:302] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837c11167a240e1a0bb6447c49c2fa9, 7d561ae0a12c473d8d2b66d42690534a; no voters:
I20260619 06:50:56.998543 23194 raft_consensus.cc:2781] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:56.998701 23194 raft_consensus.cc:482] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:56.998839 23194 raft_consensus.cc:3037] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:57.000066 23196 leader_election.cc:288] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d561ae0a12c473d8d2b66d42690534a (127.18.39.66:38291), 1837c11167a240e1a0bb6447c49c2fa9 (127.18.39.67:44897)
I20260619 06:50:57.002633 23194 raft_consensus.cc:504] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:57.003391 23194 leader_election.cc:288] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 915135f0c7b3459eb7b654fe6e973f8f (127.18.39.65:33343), 7d561ae0a12c473d8d2b66d42690534a (127.18.39.66:38291)
I20260619 06:50:57.003975 22860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "1837c11167a240e1a0bb6447c49c2fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "915135f0c7b3459eb7b654fe6e973f8f"
I20260619 06:50:57.004176 22993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "1837c11167a240e1a0bb6447c49c2fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d561ae0a12c473d8d2b66d42690534a"
I20260619 06:50:57.004253 22860 raft_consensus.cc:3037] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:57.004402 22993 raft_consensus.cc:3037] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:57.007733 22860 raft_consensus.cc:2454] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837c11167a240e1a0bb6447c49c2fa9 in term 1.
I20260619 06:50:57.007905 22993 raft_consensus.cc:2454] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1837c11167a240e1a0bb6447c49c2fa9 in term 1.
I20260619 06:50:57.008168 23061 leader_election.cc:302] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1837c11167a240e1a0bb6447c49c2fa9, 915135f0c7b3459eb7b654fe6e973f8f; no voters:
I20260619 06:50:57.008457 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.999170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:43979 (local address 127.18.39.66:38291)
0619 06:50:56.999624 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.999630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.000330 (+ 700us) server_negotiation.cc:407] Connection header received
0619 06:50:57.000632 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.000638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.000707 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.000753 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:57.001571 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.002711 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.005695 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.007070 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.007212 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.007281 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.008010 (+ 729us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.008146 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348}
I20260619 06:50:57.008462 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.999451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46973 (local address 127.18.39.67:44897)
0619 06:50:56.999624 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:56.999630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.000812 (+ 1182us) server_negotiation.cc:407] Connection header received
0619 06:50:57.000974 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.000978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.001075 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.001174 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:50:57.001833 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.002981 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.005770 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.007044 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.007174 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.007246 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.008004 (+ 758us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.008145 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 06:50:57.008788 23194 raft_consensus.cc:2781] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:57.009284 23199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.999140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:38291 (local address 127.18.39.65:43979)
0619 06:50:57.000194 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:57.000219 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:57.000460 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:57.000984 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:57.000988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:57.001014 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:57.001435 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:57.001449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.002855 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.002863 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:57.005482 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:57.005497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.007509 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.007518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:57.007780 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.007849 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 06:50:57.008616 (+ 767us) client_negotiation.cc:240] Negotiation successful
0619 06:50:57.008769 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":131,"thread_start_us":645,"threads_started":1}
I20260619 06:50:57.009270 23194 raft_consensus.cc:686] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 1 LEADER]: Becoming Leader. State: Replica: 1837c11167a240e1a0bb6447c49c2fa9, State: Running, Role: LEADER
I20260619 06:50:57.009476 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:56.999169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44897 (local address 127.18.39.65:46973)
0619 06:50:57.000693 (+ 1524us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:57.000710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:57.000831 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:57.001210 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:57.001214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:57.001239 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:50:57.001660 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:57.001671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.003162 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.003167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:57.005581 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:57.005596 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.007475 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.007484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:57.007771 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.007838 (+ 67us) client_negotiation.cc:769] Sending connection context
0619 06:50:57.008203 (+ 365us) client_negotiation.cc:240] Negotiation successful
0619 06:50:57.008959 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1393,"mutex_wait_us":70,"thread_start_us":1207,"threads_started":1}
I20260619 06:50:57.009986 22993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "915135f0c7b3459eb7b654fe6e973f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d561ae0a12c473d8d2b66d42690534a" is_pre_election: true
I20260619 06:50:57.010032 23194 consensus_queue.cc:237] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } }
I20260619 06:50:57.010315 22993 raft_consensus.cc:2379] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 915135f0c7b3459eb7b654fe6e973f8f in current term 1: Already voted for candidate 1837c11167a240e1a0bb6447c49c2fa9 in this term.
I20260619 06:50:57.010627 23125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86253a86fd9d4aa98e8296bd939f4ea5" candidate_uuid: "915135f0c7b3459eb7b654fe6e973f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1837c11167a240e1a0bb6447c49c2fa9" is_pre_election: true
I20260619 06:50:57.011695 22793 leader_election.cc:302] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 915135f0c7b3459eb7b654fe6e973f8f; no voters: 1837c11167a240e1a0bb6447c49c2fa9, 7d561ae0a12c473d8d2b66d42690534a
I20260619 06:50:57.012115 23196 raft_consensus.cc:2726] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 06:50:57.015013 22717 catalog_manager.cc:5495] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1837c11167a240e1a0bb6447c49c2fa9 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "1837c11167a240e1a0bb6447c49c2fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1837c11167a240e1a0bb6447c49c2fa9" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44897 } health_report { overall_health: HEALTHY } } }
I20260619 06:50:57.103243 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:57.097300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.18.39.67:44897)
0619 06:50:57.097509 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:57.097516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.098170 (+ 654us) server_negotiation.cc:407] Connection header received
0619 06:50:57.098383 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.098387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.098474 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.098597 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:57.098963 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.099622 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.101002 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.101445 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.101540 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.101956 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:57.102588 (+ 632us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:57.102724 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.102859 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.102923 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:50:57.111074 22993 raft_consensus.cc:1261] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 1 FOLLOWER]: Refusing update from remote peer 1837c11167a240e1a0bb6447c49c2fa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:57.111116 22860 raft_consensus.cc:1261] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 1 FOLLOWER]: Refusing update from remote peer 1837c11167a240e1a0bb6447c49c2fa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:57.111996 23201 consensus_queue.cc:1035] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "915135f0c7b3459eb7b654fe6e973f8f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:57.112355 23194 consensus_queue.cc:1035] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d561ae0a12c473d8d2b66d42690534a" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 38291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:57.123540 23205 mvcc.cc:205] Tried to move back new op lower bound from 7298465206716715008 to 7298465206318469120. Current Snapshot: MvccSnapshot[applied={T|T < 7298465206716715008}]
I20260619 06:50:57.141150 22717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:57.141489 22717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:50:57.145241 22717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce18cf9951854bb78c7933654d0ac07b: Sending DeleteTablet for 3 replicas of tablet 86253a86fd9d4aa98e8296bd939f4ea5
I20260619 06:50:57.146631 22973 tablet_service.cc:1514] Processing DeleteTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38556
I20260619 06:50:57.146638 23105 tablet_service.cc:1514] Processing DeleteTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38116
I20260619 06:50:57.147030 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 86253a86fd9d4aa98e8296bd939f4ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:50:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38118
I20260619 06:50:57.148378 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22778
I20260619 06:50:57.160496 23216 tablet_replica.cc:330] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: stopping tablet replica
I20260619 06:50:57.160666 23217 tablet_replica.cc:330] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: stopping tablet replica
I20260619 06:50:57.161171 23217 raft_consensus.cc:2229] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:57.161859 23217 raft_consensus.cc:2258] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:57.162595 23216 raft_consensus.cc:2229] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:50:57.163396 23216 raft_consensus.cc:2258] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:57.164090 23217 ts_tablet_manager.cc:1905] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:57.165269 23216 ts_tablet_manager.cc:1905] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:57.167564 23218 tablet_replica.cc:330] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: stopping tablet replica
I20260619 06:50:57.168012 23218 raft_consensus.cc:2229] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:50:57.168534 23218 raft_consensus.cc:2258] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:50:57.171561 23217 ts_tablet_manager.cc:1918] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:57.171774 23217 log.cc:1196] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/86253a86fd9d4aa98e8296bd939f4ea5
I20260619 06:50:57.172277 23217 ts_tablet_manager.cc:1939] T 86253a86fd9d4aa98e8296bd939f4ea5 P 7d561ae0a12c473d8d2b66d42690534a: Deleting consensus metadata
I20260619 06:50:57.172333 23218 ts_tablet_manager.cc:1905] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:50:57.173557 23216 ts_tablet_manager.cc:1918] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:57.173753 23216 log.cc:1196] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/86253a86fd9d4aa98e8296bd939f4ea5
I20260619 06:50:57.174237 22704 catalog_manager.cc:4841] TS 7d561ae0a12c473d8d2b66d42690534a (127.18.39.66:38291): tablet 86253a86fd9d4aa98e8296bd939f4ea5 (table test-table [id=0df2bf3f88254d5b920a14e3865d1982]) successfully deleted
I20260619 06:50:57.174564 23216 ts_tablet_manager.cc:1939] T 86253a86fd9d4aa98e8296bd939f4ea5 P 1837c11167a240e1a0bb6447c49c2fa9: Deleting consensus metadata
I20260619 06:50:57.176273 22701 catalog_manager.cc:4841] TS 1837c11167a240e1a0bb6447c49c2fa9 (127.18.39.67:44897): tablet 86253a86fd9d4aa98e8296bd939f4ea5 (table test-table [id=0df2bf3f88254d5b920a14e3865d1982]) successfully deleted
Jun 19 06:50:57 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:50:57.179368 23218 ts_tablet_manager.cc:1918] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:50:57.179541 23218 log.cc:1196] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/86253a86fd9d4aa98e8296bd939f4ea5
I20260619 06:50:57.180028 23218 ts_tablet_manager.cc:1939] T 86253a86fd9d4aa98e8296bd939f4ea5 P 915135f0c7b3459eb7b654fe6e973f8f: Deleting consensus metadata
I20260619 06:50:57.181963 22704 catalog_manager.cc:4841] TS 915135f0c7b3459eb7b654fe6e973f8f (127.18.39.65:33343): tablet 86253a86fd9d4aa98e8296bd939f4ea5 (table test-table [id=0df2bf3f88254d5b920a14e3865d1982]) successfully deleted
I20260619 06:50:57.185250 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:57.150062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.18.39.65:33343)
0619 06:50:57.150243 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:57.150250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.159607 (+ 9357us) server_negotiation.cc:407] Connection header received
0619 06:50:57.160107 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.160115 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.160202 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.160281 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:57.162986 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.164536 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.168946 (+ 4410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.169626 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.169765 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.178880 (+ 9115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:57.178892 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:57.178907 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:57.178948 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:57.182481 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.183191 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.183200 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.183203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.183344 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.183727 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.183732 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.183736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.184066 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:57.184366 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.184663 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.184911 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:50:57.185909 22900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:57.297264 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22778
I20260619 06:50:57.315526 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22911
Jun 19 06:50:57 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:50:57.331472 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:57.316384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.18.39.66:38291)
0619 06:50:57.316775 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:57.316779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.316793 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:50:57.316844 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.316848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.316903 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.317025 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:57.318331 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.319057 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.320291 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.320724 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.320864 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.325254 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:57.325270 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:57.325287 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:57.325327 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:57.328326 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.329042 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.329050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.329052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.329182 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.329888 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.329894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.329898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.330265 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:57.330593 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.330899 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.331125 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260619 06:50:57.332357 23033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:57.422205 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22911
I20260619 06:50:57.438499 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23043
Jun 19 06:50:57 dist-test-slave-1h5s krb5kdc[22673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:50:57.459000 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:57.439317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.18.39.67:44897)
0619 06:50:57.439513 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:57.439517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:57.439534 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:50:57.439747 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:57.439751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:57.439815 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:57.439908 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:57.441743 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.442494 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.444305 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:57.444759 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:57.444856 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:57.450823 (+ 5967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:57.450836 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:57.450854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:57.450915 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:57.455617 (+ 4702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.456541 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.456551 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.456555 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.456735 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:57.457164 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:57.457169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:57.457173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:57.457763 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:57.458110 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:57.458391 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 06:50:57.458631 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:50:57.459781 23165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:57.583137 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23043
I20260619 06:50:57.602425 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 22686
I20260619 06:50:57.603516 22747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:57.712349 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 22686
2026-06-19T06:50:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4657 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T06:50:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:57Z Disabled control of system clock
I20260619 06:50:57.779482 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:45523
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:45523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 06:50:58.104216 23227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:58.104545 23227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:58.104709 23227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:58.112406 23227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:58.112532 23227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:58.112627 23227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:58.112742 23227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:58.112818 23227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:58.122440 23227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:45523
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:58.123782 23227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:58.125633 23227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:58.137773 23233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:58.137924 23235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:58.137795 23232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:58.138701 23227 server_base.cc:1034] running on GCE node
I20260619 06:50:58.139531 23227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:58.141029 23227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:58.142256 23227 hybrid_clock.cc:648] HybridClock initialized: now 1781851858142211 us; error 64 us; skew 500 ppm
I20260619 06:50:58.142624 23227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:50:58.145054 23227 webserver.cc:460] Webserver started at http://127.18.39.126:45381/ using document root <none> and password file <none>
I20260619 06:50:58.145889 23227 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:58.146019 23227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:58.146358 23227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:58.149228 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "88e9bc63601a4d64accdbe8706827858"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "755567b0f723a6cbabe801273d127439"
server_key_iv: "32eb763b83d81ec621886e46d69ad3ae"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.150095 23227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "88e9bc63601a4d64accdbe8706827858"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "755567b0f723a6cbabe801273d127439"
server_key_iv: "32eb763b83d81ec621886e46d69ad3ae"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.155313 23227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:58.158783 23241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:58.160133 23227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:50:58.160324 23227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "88e9bc63601a4d64accdbe8706827858"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "755567b0f723a6cbabe801273d127439"
server_key_iv: "32eb763b83d81ec621886e46d69ad3ae"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.160563 23227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:58.166458 23227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:58.167248 23227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:58.167559 23227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:58.186187 23227 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:45523
I20260619 06:50:58.186262 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:45523 every 8 connection(s)
I20260619 06:50:58.187618 23227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:58.188704 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23227
I20260619 06:50:58.188922 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:50:58.189791 18589 external_mini_cluster.cc:1428] Setting key 5f7f4d9add098ce181c22b0d17385e13
I20260619 06:50:58.192809 23293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:58.198734 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:58.192030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.18.39.126:45523)
0619 06:50:58.193311 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:58.193323 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:58.193438 (+ 115us) server_negotiation.cc:407] Connection header received
0619 06:50:58.195043 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:58.195063 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:58.195498 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:58.195835 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:58.196993 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:58.197000 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:58.197012 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:58.197026 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:58.197105 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:58.197124 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:58.197297 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:58.197498 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 06:50:58.197807 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":425,"threads_started":1}
I20260619 06:50:58.204869 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: Bootstrap starting.
I20260619 06:50:58.207878 23293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:58.208882 23293 log.cc:824] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:58.211979 23293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: No bootstrap required, opened a new log
I20260619 06:50:58.219677 23293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } }
I20260619 06:50:58.220112 23293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:58.220232 23293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e9bc63601a4d64accdbe8706827858, State: Initialized, Role: FOLLOWER
I20260619 06:50:58.220821 23293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } }
I20260619 06:50:58.221076 23293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:58.221230 23293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:58.221457 23293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:58.223634 23293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } }
I20260619 06:50:58.224167 23293 leader_election.cc:302] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e9bc63601a4d64accdbe8706827858; no voters:
I20260619 06:50:58.224774 23293 leader_election.cc:288] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:58.224917 23298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:58.226121 23298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [term 1 LEADER]: Becoming Leader. State: Replica: 88e9bc63601a4d64accdbe8706827858, State: Running, Role: LEADER
I20260619 06:50:58.226773 23298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } }
I20260619 06:50:58.227330 23293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:58.229651 23299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e9bc63601a4d64accdbe8706827858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } } }
I20260619 06:50:58.230058 23299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:58.230147 23300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e9bc63601a4d64accdbe8706827858. Latest consensus state: current_term: 1 leader_uuid: "88e9bc63601a4d64accdbe8706827858" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e9bc63601a4d64accdbe8706827858" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 45523 } } }
I20260619 06:50:58.230558 23300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:58.232146 23307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:58.236774 23307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:58.245111 23307 catalog_manager.cc:1347] Generated new cluster ID: 501d9c06efd24459bc9b66ceaf508600
I20260619 06:50:58.245229 23307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:58.280926 23307 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:58.281471 23307 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:58.301100 23307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: Generated new TSK 0
I20260619 06:50:58.302228 23307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:58.315129 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--builtin_ntp_servers=127.18.39.84:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 06:50:58.649608 23317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:58.649925 23317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:58.650085 23317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:58.657605 23317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:58.657743 23317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:58.657917 23317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:50:58.667753 23317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:58.669065 23317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:58.670883 23317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:58.684087 23325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:58.684132 23323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:58.684044 23322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:58.685129 23317 server_base.cc:1034] running on GCE node
I20260619 06:50:58.685839 23317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:58.686827 23317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:58.688055 23317 hybrid_clock.cc:648] HybridClock initialized: now 1781851858688022 us; error 55 us; skew 500 ppm
I20260619 06:50:58.688375 23317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:50:58.690809 23317 webserver.cc:460] Webserver started at http://127.18.39.65:36255/ using document root <none> and password file <none>
I20260619 06:50:58.691540 23317 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:58.691673 23317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:58.692011 23317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:58.694909 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "cf88939c627d483588417792c1133be5"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "c3a344f0d6185e58b82669b80079451b"
server_key_iv: "84fa0f4b6e5272fdae270c3dce0acf07"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.695706 23317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "cf88939c627d483588417792c1133be5"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "c3a344f0d6185e58b82669b80079451b"
server_key_iv: "84fa0f4b6e5272fdae270c3dce0acf07"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.700778 23317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:58.704428 23331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:58.705914 23317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:58.706094 23317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "cf88939c627d483588417792c1133be5"
format_stamp: "Formatted at 2026-06-19 06:50:58 on dist-test-slave-1h5s"
server_key: "c3a344f0d6185e58b82669b80079451b"
server_key_iv: "84fa0f4b6e5272fdae270c3dce0acf07"
server_key_version: "encryptionkey@0"
I20260619 06:50:58.706321 23317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:58.711962 23317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:58.712757 23317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:58.713042 23317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:58.713994 23317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:58.715423 23317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:58.715538 23317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:58.715672 23317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:58.715755 23317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:58.748939 23317 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41329
I20260619 06:50:58.748986 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41329 every 8 connection(s)
I20260619 06:50:58.750391 23317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:50:58.756435 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23317
I20260619 06:50:58.756665 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:50:58.757503 18589 external_mini_cluster.cc:1428] Setting key e9896edafc327472920c43922a536f31
I20260619 06:50:58.759166 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:58.752929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:59159 (local address 127.18.39.126:45523)
0619 06:50:58.753426 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:58.753431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:58.754539 (+ 1108us) server_negotiation.cc:407] Connection header received
0619 06:50:58.756338 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:58.756343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:58.756396 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:58.756494 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:58.758208 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:58.758214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:58.758219 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:58.758241 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:58.758293 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:58.758310 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:58.758446 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:58.758783 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 06:50:58.758851 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260619 06:50:58.760215 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:58.753507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:45523 (local address 127.18.39.65:59159)
0619 06:50:58.754381 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:58.754424 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:58.756009 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:58.756783 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:58.756794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:58.756926 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:58.757779 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:58.757782 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:58.757832 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:58.757955 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:58.758533 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:58.758544 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:58.758552 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:50:58.758828 (+ 276us) client_negotiation.cc:240] Negotiation successful
0619 06:50:58.759039 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":267,"threads_started":1}
I20260619 06:50:58.760856 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--builtin_ntp_servers=127.18.39.84:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 06:50:58.761831 23444 heartbeater.cc:347] Connected to a master server at 127.18.39.126:45523
I20260619 06:50:58.762145 23444 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:58.762737 23444 heartbeater.cc:511] Master 127.18.39.126:45523 requested a full tablet report, sending...
I20260619 06:50:58.764835 23258 ts_manager.cc:194] Registered new tserver with Master: cf88939c627d483588417792c1133be5 (127.18.39.65:41329)
W20260619 06:50:59.089046 23448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:59.089385 23448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:59.089532 23448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:59.096918 23448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:59.097038 23448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:59.097146 23448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:50:59.106431 23448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:59.107712 23448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:59.109477 23448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:59.122748 23456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:59.122779 23453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:59.122852 23454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:59.128754 23448 server_base.cc:1034] running on GCE node
I20260619 06:50:59.129519 23448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:59.130563 23448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:59.131785 23448 hybrid_clock.cc:648] HybridClock initialized: now 1781851859131753 us; error 60 us; skew 500 ppm
I20260619 06:50:59.132115 23448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:50:59.134701 23448 webserver.cc:460] Webserver started at http://127.18.39.66:33841/ using document root <none> and password file <none>
I20260619 06:50:59.135437 23448 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:59.135578 23448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:59.135911 23448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:59.138952 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "0b41e056547e4c0e88580b1eb496381c"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6b7706ffaa2b1cb1a69f9fe331e2aafb"
server_key_iv: "2f083d698e2c4530ea9846ab96ae11ed"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.139859 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "0b41e056547e4c0e88580b1eb496381c"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6b7706ffaa2b1cb1a69f9fe331e2aafb"
server_key_iv: "2f083d698e2c4530ea9846ab96ae11ed"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.145614 23448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:50:59.149608 23462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:59.151079 23448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:50:59.151320 23448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "0b41e056547e4c0e88580b1eb496381c"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6b7706ffaa2b1cb1a69f9fe331e2aafb"
server_key_iv: "2f083d698e2c4530ea9846ab96ae11ed"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.151594 23448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:59.157692 23448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:59.158540 23448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:59.158833 23448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:59.159791 23448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:59.161311 23448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:59.161521 23448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260619 06:50:59.161669 23448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:59.161763 23448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:59.198321 23448 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:34691
I20260619 06:50:59.198364 23574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:34691 every 8 connection(s)
I20260619 06:50:59.199707 23448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:50:59.201112 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23448
I20260619 06:50:59.201309 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:50:59.201830 18589 external_mini_cluster.cc:1428] Setting key 415d2cd58001369b8cb5b5c91bc880d1
I20260619 06:50:59.205322 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--builtin_ntp_servers=127.18.39.84:39525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 06:50:59.209548 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.203664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:44013 (local address 127.18.39.126:45523)
0619 06:50:59.203938 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.203945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.204949 (+ 1004us) server_negotiation.cc:407] Connection header received
0619 06:50:59.207116 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.207122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.207207 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.207329 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.208725 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.208732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.208745 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.208765 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.208813 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.208826 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.208955 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.209168 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.209237 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260619 06:50:59.210878 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.203628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:45523 (local address 127.18.39.66:44013)
0619 06:50:59.204789 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.204844 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.206845 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.207571 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.207580 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.207676 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.208421 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.208424 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.208463 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.208552 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.208928 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.208938 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.208946 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.209264 (+ 318us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.209559 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":354,"threads_started":1}
I20260619 06:50:59.212332 23575 heartbeater.cc:347] Connected to a master server at 127.18.39.126:45523
I20260619 06:50:59.212646 23575 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:59.213269 23575 heartbeater.cc:511] Master 127.18.39.126:45523 requested a full tablet report, sending...
I20260619 06:50:59.214782 23258 ts_manager.cc:194] Registered new tserver with Master: 0b41e056547e4c0e88580b1eb496381c (127.18.39.66:34691)
W20260619 06:50:59.544344 23578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:59.544688 23578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:59.544857 23578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:59.552416 23578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:59.552578 23578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:59.552769 23578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:50:59.562561 23578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:45523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:59.564026 23578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:59.565990 23578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:59.582346 23583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:59.582803 23584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:59.583702 23578 server_base.cc:1034] running on GCE node
W20260619 06:50:59.583866 23586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:59.584641 23578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:59.585783 23578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:59.587010 23578 hybrid_clock.cc:648] HybridClock initialized: now 1781851859586957 us; error 49 us; skew 500 ppm
I20260619 06:50:59.587337 23578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:50:59.589823 23578 webserver.cc:460] Webserver started at http://127.18.39.67:39867/ using document root <none> and password file <none>
I20260619 06:50:59.590583 23578 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:59.590696 23578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:59.591058 23578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:59.594043 23578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6ed48ede36415d9cda8dbeaffef15d51"
server_key_iv: "987c1d32db2b6b486d028fc9a8c8862d"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.594856 23578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6ed48ede36415d9cda8dbeaffef15d51"
server_key_iv: "987c1d32db2b6b486d028fc9a8c8862d"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.600314 23578 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:50:59.604018 23592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:59.605647 23578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:59.605815 23578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6"
format_stamp: "Formatted at 2026-06-19 06:50:59 on dist-test-slave-1h5s"
server_key: "6ed48ede36415d9cda8dbeaffef15d51"
server_key_iv: "987c1d32db2b6b486d028fc9a8c8862d"
server_key_version: "encryptionkey@0"
I20260619 06:50:59.606048 23578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:59.612214 23578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:59.613022 23578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:59.613286 23578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:59.614338 23578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:50:59.615823 23578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:50:59.615912 23578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:59.616075 23578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:50:59.616153 23578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:59.662213 23578 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:34573
I20260619 06:50:59.662253 23704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:34573 every 8 connection(s)
I20260619 06:50:59.663547 23578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:50:59.668206 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23578
I20260619 06:50:59.668395 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:50:59.669044 18589 external_mini_cluster.cc:1428] Setting key 44fea4f41c6b77b6f0a79485d4db777b
I20260619 06:50:59.673707 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.666602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60207 (local address 127.18.39.126:45523)
0619 06:50:59.666778 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.666782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.668589 (+ 1807us) server_negotiation.cc:407] Connection header received
0619 06:50:59.670989 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.670998 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.671085 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.671218 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.672849 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.672857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.672865 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.672891 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.672937 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.672957 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.673087 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.673255 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.673325 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:50:59.674580 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.667401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:45523 (local address 127.18.39.67:60207)
0619 06:50:59.668372 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.668433 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.670653 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.671514 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.671526 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.671660 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.672507 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.672511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.672564 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.672665 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.673061 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.673068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.673073 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.673289 (+ 216us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.673575 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":2118,"threads_started":1}
I20260619 06:50:59.675951 23705 heartbeater.cc:347] Connected to a master server at 127.18.39.126:45523
I20260619 06:50:59.676265 23705 heartbeater.cc:464] Registering TS with master...
I20260619 06:50:59.676868 23705 heartbeater.cc:511] Master 127.18.39.126:45523 requested a full tablet report, sending...
I20260619 06:50:59.678368 23258 ts_manager.cc:194] Registered new tserver with Master: ba5f1fd66cd84ebf8bd2029494fbadf6 (127.18.39.67:34573)
I20260619 06:50:59.684849 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:50:59.692613 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.689713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.18.39.126:45523)
0619 06:50:59.689942 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.689951 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.690492 (+ 541us) server_negotiation.cc:407] Connection header received
0619 06:50:59.690661 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.690669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.690755 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.690903 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.691899 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.691908 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.691916 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.691939 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.691984 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.692003 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.692141 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.692192 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.692256 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:50:59.696794 23258 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:50:59.703526 23258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:50:59.733601 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.717293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.18.39.65:41329)
0619 06:50:59.730020 (+ 12727us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.730034 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.730892 (+ 858us) server_negotiation.cc:407] Connection header received
0619 06:50:59.731040 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.731052 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.731311 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.731502 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.732728 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.732737 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.732751 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.732767 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.732818 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.732832 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.733034 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.733104 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.733202 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12455,"thread_start_us":12724,"threads_started":1}
I20260619 06:50:59.733623 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.716665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44282)
0619 06:50:59.730727 (+ 14062us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.730745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.730906 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.731519 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.731526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.731637 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.732457 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.732460 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.732507 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.732598 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.732931 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.732941 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.732949 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.733118 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.733202 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13911,"thread_start_us":4833,"threads_started":1}
I20260619 06:50:59.735962 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.722854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:34691 (local address 127.0.0.1:48342)
0619 06:50:59.733850 (+ 10996us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.733867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.733953 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.734498 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.734502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.734535 (+ 33us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.735124 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.735127 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.735143 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.735190 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.735678 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.735682 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.735685 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.735752 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.735790 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10896,"thread_start_us":7570,"threads_started":1}
I20260619 06:50:59.736174 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.722875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.18.39.66:34691)
0619 06:50:59.733758 (+ 10883us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.733769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.733933 (+ 164us) server_negotiation.cc:407] Connection header received
0619 06:50:59.734062 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.734072 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.734320 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.734505 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.735452 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.735464 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.735477 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.735506 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.735560 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.735580 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.735713 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.735777 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.735869 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10715,"thread_start_us":10598,"threads_started":1}
I20260619 06:50:59.738590 23379 tablet_service.cc:1467] Processing CreateTablet for tablet af06ae30b1fc40159a550d8b6558b203 (DEFAULT_TABLE table=test-table [id=e688f9afe75c428a930f6e334643b2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:59.740247 23379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af06ae30b1fc40159a550d8b6558b203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:59.739953 23510 tablet_service.cc:1467] Processing CreateTablet for tablet af06ae30b1fc40159a550d8b6558b203 (DEFAULT_TABLE table=test-table [id=e688f9afe75c428a930f6e334643b2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:59.741417 23510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af06ae30b1fc40159a550d8b6558b203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:59.757550 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.716506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:34573 (local address 127.0.0.1:51440)
0619 06:50:59.721995 (+ 5489us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.722055 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.722219 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.754263 (+ 32044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.754271 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.754301 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.755173 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.755177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.755199 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.755263 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.756884 (+ 1621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.756889 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.756895 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.757027 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.757088 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5163,"thread_start_us":38684,"threads_started":1}
I20260619 06:50:59.759820 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.717100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.18.39.67:34573)
0619 06:50:59.753416 (+ 36316us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.753429 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.753498 (+ 69us) server_negotiation.cc:407] Connection header received
0619 06:50:59.753607 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.753619 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.754004 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.754253 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.756618 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.756628 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.756644 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.756665 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.756728 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.756752 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.757995 (+ 1243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.758065 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.758172 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36085,"thread_start_us":35916,"threads_started":1}
I20260619 06:50:59.761854 23723 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Bootstrap starting.
I20260619 06:50:59.764600 23640 tablet_service.cc:1467] Processing CreateTablet for tablet af06ae30b1fc40159a550d8b6558b203 (DEFAULT_TABLE table=test-table [id=e688f9afe75c428a930f6e334643b2c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:50:59.766578 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af06ae30b1fc40159a550d8b6558b203. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:59.768115 23444 heartbeater.cc:503] Master 127.18.39.126:45523 was elected leader, sending a full tablet report...
I20260619 06:50:59.769883 23723 tablet_bootstrap.cc:654] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:59.771565 23723 log.cc:824] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:59.780890 23724 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Bootstrap starting.
I20260619 06:50:59.783944 23723 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: No bootstrap required, opened a new log
I20260619 06:50:59.784255 23723 ts_tablet_manager.cc:1397] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Time spent bootstrapping tablet: real 0.023s user 0.009s sys 0.006s
I20260619 06:50:59.785043 23724 tablet_bootstrap.cc:654] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:59.786371 23724 log.cc:824] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:59.789196 23726 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Bootstrap starting.
I20260619 06:50:59.792862 23726 tablet_bootstrap.cc:654] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:59.795518 23726 log.cc:824] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:59.795831 23724 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: No bootstrap required, opened a new log
I20260619 06:50:59.796204 23724 ts_tablet_manager.cc:1397] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260619 06:50:59.797915 23723 raft_consensus.cc:348] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.798408 23723 raft_consensus.cc:374] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:59.798559 23723 raft_consensus.cc:729] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf88939c627d483588417792c1133be5, State: Initialized, Role: FOLLOWER
I20260619 06:50:59.799245 23723 consensus_queue.cc:260] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.807123 23723 ts_tablet_manager.cc:1428] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Time spent starting tablet: real 0.023s user 0.019s sys 0.004s
I20260619 06:50:59.809103 23726 tablet_bootstrap.cc:492] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: No bootstrap required, opened a new log
I20260619 06:50:59.808881 23724 raft_consensus.cc:348] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.809485 23724 raft_consensus.cc:374] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:59.809494 23726 ts_tablet_manager.cc:1397] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.008s
I20260619 06:50:59.809613 23724 raft_consensus.cc:729] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b41e056547e4c0e88580b1eb496381c, State: Initialized, Role: FOLLOWER
I20260619 06:50:59.810401 23724 consensus_queue.cc:260] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.811684 23575 heartbeater.cc:503] Master 127.18.39.126:45523 was elected leader, sending a full tablet report...
I20260619 06:50:59.812343 23724 ts_tablet_manager.cc:1428] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Time spent starting tablet: real 0.016s user 0.011s sys 0.003s
I20260619 06:50:59.818434 23726 raft_consensus.cc:348] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.818989 23726 raft_consensus.cc:374] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:59.819100 23726 raft_consensus.cc:729] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba5f1fd66cd84ebf8bd2029494fbadf6, State: Initialized, Role: FOLLOWER
I20260619 06:50:59.819778 23726 consensus_queue.cc:260] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.820742 23705 heartbeater.cc:503] Master 127.18.39.126:45523 was elected leader, sending a full tablet report...
I20260619 06:50:59.821214 23726 ts_tablet_manager.cc:1428] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
W20260619 06:50:59.915663 23706 tablet.cc:2369] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:59.947763 23730 raft_consensus.cc:482] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:50:59.948096 23730 raft_consensus.cc:504] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.949894 23730 leader_election.cc:288] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba5f1fd66cd84ebf8bd2029494fbadf6 (127.18.39.67:34573), cf88939c627d483588417792c1133be5 (127.18.39.65:41329)
W20260619 06:50:59.951989 23576 tablet.cc:2369] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:59.953379 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.950233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:40453 (local address 127.18.39.67:34573)
0619 06:50:59.950426 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.950434 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.951298 (+ 864us) server_negotiation.cc:407] Connection header received
0619 06:50:59.951388 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.951395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.951462 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.951561 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.952717 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.952723 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.952728 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.952753 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.952782 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.952794 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.952939 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.953014 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.953089 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:50:59.953555 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.950531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:56907 (local address 127.18.39.65:41329)
0619 06:50:59.950721 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:59.950726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:59.951446 (+ 720us) server_negotiation.cc:407] Connection header received
0619 06:50:59.951579 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:59.951583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:59.951641 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:59.951752 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:59.952909 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:59.952914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:59.952918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:59.952938 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:59.952975 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:59.952993 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:59.953116 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:59.953211 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 06:50:59.953264 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:50:59.953632 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.950130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:34573 (local address 127.18.39.66:40453)
0619 06:50:59.951113 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.951140 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.951264 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.951596 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.951601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.951622 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.952494 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.952499 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.952520 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.952572 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.952901 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.952908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.952913 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.953035 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.953110 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":481,"threads_started":1}
I20260619 06:50:59.953774 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:59.950271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.18.39.66:56907)
0619 06:50:59.951330 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:59.951348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:59.951450 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:59.951808 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:59.951813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:59.951836 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:59.952688 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:50:59.952691 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:59.952710 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:50:59.952750 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:59.953095 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:59.953099 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:59.953103 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 06:50:59.953244 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 06:50:59.953291 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"thread_start_us":612,"threads_started":1}
I20260619 06:50:59.954743 23660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af06ae30b1fc40159a550d8b6558b203" candidate_uuid: "0b41e056547e4c0e88580b1eb496381c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" is_pre_election: true
I20260619 06:50:59.954743 23399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af06ae30b1fc40159a550d8b6558b203" candidate_uuid: "0b41e056547e4c0e88580b1eb496381c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf88939c627d483588417792c1133be5" is_pre_election: true
I20260619 06:50:59.955341 23399 raft_consensus.cc:2454] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b41e056547e4c0e88580b1eb496381c in term 0.
I20260619 06:50:59.955363 23660 raft_consensus.cc:2454] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b41e056547e4c0e88580b1eb496381c in term 0.
I20260619 06:50:59.956058 23466 leader_election.cc:302] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b41e056547e4c0e88580b1eb496381c, cf88939c627d483588417792c1133be5; no voters:
I20260619 06:50:59.956424 23730 raft_consensus.cc:2781] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:50:59.956578 23730 raft_consensus.cc:482] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:50:59.956681 23730 raft_consensus.cc:3037] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:59.958854 23730 raft_consensus.cc:504] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.959414 23730 leader_election.cc:288] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [CANDIDATE]: Term 1 election: Requested vote from peers ba5f1fd66cd84ebf8bd2029494fbadf6 (127.18.39.67:34573), cf88939c627d483588417792c1133be5 (127.18.39.65:41329)
I20260619 06:50:59.959901 23660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af06ae30b1fc40159a550d8b6558b203" candidate_uuid: "0b41e056547e4c0e88580b1eb496381c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6"
I20260619 06:50:59.959959 23399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af06ae30b1fc40159a550d8b6558b203" candidate_uuid: "0b41e056547e4c0e88580b1eb496381c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf88939c627d483588417792c1133be5"
I20260619 06:50:59.960127 23660 raft_consensus.cc:3037] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:59.960203 23399 raft_consensus.cc:3037] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:59.962424 23660 raft_consensus.cc:2454] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b41e056547e4c0e88580b1eb496381c in term 1.
I20260619 06:50:59.962495 23399 raft_consensus.cc:2454] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b41e056547e4c0e88580b1eb496381c in term 1.
I20260619 06:50:59.962877 23463 leader_election.cc:302] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b41e056547e4c0e88580b1eb496381c, ba5f1fd66cd84ebf8bd2029494fbadf6; no voters:
I20260619 06:50:59.963184 23730 raft_consensus.cc:2781] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:59.963636 23730 raft_consensus.cc:686] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 1 LEADER]: Becoming Leader. State: Replica: 0b41e056547e4c0e88580b1eb496381c, State: Running, Role: LEADER
I20260619 06:50:59.964216 23730 consensus_queue.cc:237] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } }
I20260619 06:50:59.968742 23258 catalog_manager.cc:5495] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b41e056547e4c0e88580b1eb496381c (127.18.39.66). New cstate: current_term: 1 leader_uuid: "0b41e056547e4c0e88580b1eb496381c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b41e056547e4c0e88580b1eb496381c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 34691 } health_report { overall_health: HEALTHY } } }
I20260619 06:51:00.001992 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.000025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.18.39.66:34691)
0619 06:51:00.000204 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:00.000208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:00.000226 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:00.000425 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:00.000429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:00.000510 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:00.000597 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:00.001470 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:00.001474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:00.001477 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:00.001490 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:00.001520 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:00.001531 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:00.001622 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:00.001728 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 06:51:00.001772 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 06:51:00.002786 23445 tablet.cc:2369] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:51:00.010135 23399 raft_consensus.cc:1261] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 1 FOLLOWER]: Refusing update from remote peer 0b41e056547e4c0e88580b1eb496381c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:00.010135 23660 raft_consensus.cc:1261] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 1 FOLLOWER]: Refusing update from remote peer 0b41e056547e4c0e88580b1eb496381c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:00.011209 23734 consensus_queue.cc:1035] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba5f1fd66cd84ebf8bd2029494fbadf6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:00.011618 23730 consensus_queue.cc:1035] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf88939c627d483588417792c1133be5" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:00.043855 23258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:00.044127 23258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:00.047453 23258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 88e9bc63601a4d64accdbe8706827858: Sending DeleteTablet for 3 replicas of tablet af06ae30b1fc40159a550d8b6558b203
I20260619 06:51:00.048700 23640 tablet_service.cc:1514] Processing DeleteTablet for tablet af06ae30b1fc40159a550d8b6558b203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:00 UTC) from {username='slave'} at 127.0.0.1:51440
I20260619 06:51:00.048727 23379 tablet_service.cc:1514] Processing DeleteTablet for tablet af06ae30b1fc40159a550d8b6558b203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:00 UTC) from {username='slave'} at 127.0.0.1:44282
I20260619 06:51:00.048974 23510 tablet_service.cc:1514] Processing DeleteTablet for tablet af06ae30b1fc40159a550d8b6558b203 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:00 UTC) from {username='slave'} at 127.0.0.1:48342
I20260619 06:51:00.051564 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23317
I20260619 06:51:00.061158 23749 tablet_replica.cc:330] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: stopping tablet replica
I20260619 06:51:00.061968 23749 raft_consensus.cc:2229] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:00.062894 23749 raft_consensus.cc:2258] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:00.063843 23747 tablet_replica.cc:330] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: stopping tablet replica
I20260619 06:51:00.064358 23747 raft_consensus.cc:2229] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:00.064932 23747 raft_consensus.cc:2258] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:00.067442 23749 ts_tablet_manager.cc:1905] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:00.071223 23747 ts_tablet_manager.cc:1905] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:00.076025 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.053235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.18.39.65:41329)
0619 06:51:00.053444 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:00.053452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:00.073235 (+ 19783us) server_negotiation.cc:407] Connection header received
0619 06:51:00.073796 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:00.073802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:00.073930 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:00.074021 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:00.075199 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:00.075205 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:00.075210 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:00.075229 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:00.075263 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:00.075278 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:00.075380 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:00.075600 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 06:51:00.075665 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:51:00.076160 23749 ts_tablet_manager.cc:1918] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:00.076355 23749 log.cc:1196] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/af06ae30b1fc40159a550d8b6558b203
I20260619 06:51:00.076964 23749 ts_tablet_manager.cc:1939] T af06ae30b1fc40159a550d8b6558b203 P 0b41e056547e4c0e88580b1eb496381c: Deleting consensus metadata
I20260619 06:51:00.077200 23439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:00.079241 23245 catalog_manager.cc:4841] TS 0b41e056547e4c0e88580b1eb496381c (127.18.39.66:34691): tablet af06ae30b1fc40159a550d8b6558b203 (table test-table [id=e688f9afe75c428a930f6e334643b2c3]) successfully deleted
I20260619 06:51:00.081074 23748 tablet_replica.cc:330] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: stopping tablet replica
I20260619 06:51:00.082116 23748 raft_consensus.cc:2229] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:00.082963 23748 raft_consensus.cc:2258] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:00.085569 23748 ts_tablet_manager.cc:1905] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:00.093410 23748 ts_tablet_manager.cc:1918] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:00.093623 23748 log.cc:1196] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/af06ae30b1fc40159a550d8b6558b203
I20260619 06:51:00.094218 23748 ts_tablet_manager.cc:1939] T af06ae30b1fc40159a550d8b6558b203 P ba5f1fd66cd84ebf8bd2029494fbadf6: Deleting consensus metadata
I20260619 06:51:00.096472 23242 catalog_manager.cc:4841] TS ba5f1fd66cd84ebf8bd2029494fbadf6 (127.18.39.67:34573): tablet af06ae30b1fc40159a550d8b6558b203 (table test-table [id=e688f9afe75c428a930f6e334643b2c3]) successfully deleted
I20260619 06:51:00.176106 23747 ts_tablet_manager.cc:1918] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:00.176261 23747 log.cc:1196] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/af06ae30b1fc40159a550d8b6558b203
I20260619 06:51:00.176743 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23317
I20260619 06:51:00.176887 23747 ts_tablet_manager.cc:1939] T af06ae30b1fc40159a550d8b6558b203 P cf88939c627d483588417792c1133be5: Deleting consensus metadata
I20260619 06:51:00.201184 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23448
W20260619 06:51:00.201519 23245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.39.65:41329 (error 108)
I20260619 06:51:00.205220 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.202507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.18.39.66:34691)
0619 06:51:00.202668 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:00.202674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:00.202693 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:00.202801 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:00.202807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:00.202873 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:00.202988 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:00.204108 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:00.204114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:00.204119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:00.204139 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:00.204184 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:00.204200 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:00.204316 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:00.204849 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 06:51:00.204915 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:51:00.205966 23570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:00.207576 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.207059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44298)
0619 06:51:00.207260 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.207415 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 06:51:00.207945 23245 catalog_manager.cc:4584] TS cf88939c627d483588417792c1133be5 (127.18.39.65:41329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af06ae30b1fc40159a550d8b6558b203: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
I20260619 06:51:00.261626 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.260979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44310)
0619 06:51:00.261240 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.261427 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260619 06:51:00.262794 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.262386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44318)
0619 06:51:00.262550 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.262651 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:51:00.317508 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23448
I20260619 06:51:00.323536 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.322933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44326)
0619 06:51:00.323194 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.323314 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:51:00.324704 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.324263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44330)
0619 06:51:00.324443 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.324545 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:51:00.337385 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23578
I20260619 06:51:00.340339 23719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.338181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51452 (local address 127.18.39.67:34573)
0619 06:51:00.338377 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:00.338384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:00.338404 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:51:00.338612 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:00.338618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:00.338722 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:00.338828 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:00.339696 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:00.339702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:00.339706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:00.339724 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:00.339764 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:00.339776 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:00.339906 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:00.339989 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 06:51:00.340049 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:51:00.340989 23700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:00.405745 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.405135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44342)
0619 06:51:00.405425 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.405532 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260619 06:51:00.406899 23718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:00.406452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41329 (local address 127.0.0.1:44348)
0619 06:51:00.406683 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:00.406763 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:41329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260619 06:51:00.434157 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23578
I20260619 06:51:00.450101 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23227
I20260619 06:51:00.451306 23288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:00.557719 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23227
2026-06-19T06:51:00Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2844 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 06:51:00.598948 18589 mini_oidc.cc:152] Starting JWKS server
I20260619 06:51:00.601595 18589 webserver.cc:460] Webserver started at https://127.0.0.1:45895/ using document root <none> and password file <none>
I20260619 06:51:00.601900 18589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:51:00.602509 18589 webserver.cc:460] Webserver started at http://127.0.0.1:39673/ using document root <none> and password file <none>
2026-06-19T06:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:00Z Disabled control of system clock
I20260619 06:51:00.631675 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37099
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:44443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:37099
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45895/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/testchainca.cert with env {}
W20260619 06:51:00.956615 23761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:00.956933 23761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:00.957084 23761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:00.964644 23761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:00.964795 23761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:00.964895 23761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:00.964970 23761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:00.965030 23761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:51:00.965142 23761 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:51:00.965221 23761 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45895/jwks.json
I20260619 06:51:00.975679 23761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:37099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45895/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:00.977115 23761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:00.978566 23761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:00.991358 23766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:00.991447 23767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:00.991462 23769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:00.992305 23761 server_base.cc:1034] running on GCE node
I20260619 06:51:00.993160 23761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:00.994714 23761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:00.995965 23761 hybrid_clock.cc:648] HybridClock initialized: now 1781851860995902 us; error 58 us; skew 500 ppm
I20260619 06:51:00.996340 23761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:01.003212 23761 webserver.cc:460] Webserver started at http://127.18.39.126:35315/ using document root <none> and password file <none>
I20260619 06:51:01.004042 23761 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:01.004184 23761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:01.004499 23761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:01.008607 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "986b564fdb5d49a79c648afd5971655f"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "dcb7538e847e6cf9d946edc209559b68"
server_key_iv: "de47d868b323261e9079cfab33967d52"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.009507 23761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "986b564fdb5d49a79c648afd5971655f"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "dcb7538e847e6cf9d946edc209559b68"
server_key_iv: "de47d868b323261e9079cfab33967d52"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.014806 23761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:51:01.018399 23775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:01.019868 23761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:51:01.020078 23761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "986b564fdb5d49a79c648afd5971655f"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "dcb7538e847e6cf9d946edc209559b68"
server_key_iv: "de47d868b323261e9079cfab33967d52"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.020330 23761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:01.039168 23761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:01.040082 23761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:01.040405 23761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:01.059247 23761 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:37099
I20260619 06:51:01.059300 23826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:37099 every 8 connection(s)
I20260619 06:51:01.060742 23761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:01.064766 23827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:01.071254 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23761
I20260619 06:51:01.071496 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:01.072284 18589 external_mini_cluster.cc:1428] Setting key f69d79a4ae5446d3f36cc7e8237fb142
I20260619 06:51:01.076738 23827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f: Bootstrap starting.
I20260619 06:51:01.080008 23827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:01.081051 23827 log.cc:824] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:01.084060 23827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f: No bootstrap required, opened a new log
I20260619 06:51:01.086549 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:01.074482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.18.39.126:37099)
0619 06:51:01.075483 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:01.075494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:01.075538 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:51:01.077432 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:01.077455 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:01.077931 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:01.078340 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:01.078939 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.080329 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.081830 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.082549 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.082755 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.084010 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:01.084017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:01.084032 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:01.084653 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:01.084731 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:01.084752 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:01.084892 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:01.085122 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 06:51:01.085514 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":332,"threads_started":1}
I20260619 06:51:01.090548 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:51:01.092229 23827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } }
I20260619 06:51:01.092638 23827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:01.092748 23827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986b564fdb5d49a79c648afd5971655f, State: Initialized, Role: FOLLOWER
I20260619 06:51:01.093479 23827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } }
I20260619 06:51:01.093753 23827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:01.093897 23827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:01.094089 23827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:01.096293 23827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } }
I20260619 06:51:01.096748 23827 leader_election.cc:302] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 986b564fdb5d49a79c648afd5971655f; no voters:
I20260619 06:51:01.097237 23827 leader_election.cc:288] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:01.097422 23832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:01.098137 23832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [term 1 LEADER]: Becoming Leader. State: Replica: 986b564fdb5d49a79c648afd5971655f, State: Running, Role: LEADER
I20260619 06:51:01.098717 23832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } }
I20260619 06:51:01.099509 23827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:01.102049 23834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 986b564fdb5d49a79c648afd5971655f. Latest consensus state: current_term: 1 leader_uuid: "986b564fdb5d49a79c648afd5971655f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } } }
I20260619 06:51:01.102039 23833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "986b564fdb5d49a79c648afd5971655f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "986b564fdb5d49a79c648afd5971655f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37099 } } }
I20260619 06:51:01.102495 23834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:01.102664 23833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:01.104173 23840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:01.108953 23840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:01.117437 23840 catalog_manager.cc:1347] Generated new cluster ID: 35f8c083ef6047b58df437cee6b009ad
I20260619 06:51:01.117568 23840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:01.148043 23840 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:01.149444 23840 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:01.158344 23840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 986b564fdb5d49a79c648afd5971655f: Generated new TSK 0
I20260619 06:51:01.159222 23840 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 06:51:01.198186 23857 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 06:51:01.198393 23857 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:51:01.199608 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:01.176657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.18.39.126:37099)
0619 06:51:01.176862 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:01.176869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:01.177545 (+ 676us) server_negotiation.cc:407] Connection header received
0619 06:51:01.177832 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:01.177840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:01.177939 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:01.178119 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:51:01.178706 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.179874 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.181379 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.182084 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.182226 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.182874 (+ 648us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:51:01.198861 (+ 15987us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 06:51:01.199245 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:51:01.199745 23830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47924: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 06:51:01.201215 23856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:01.176659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:37099 (local address 127.0.0.1:47924)
0619 06:51:01.177434 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:01.177454 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:01.177614 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:01.178156 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:01.178160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:01.178171 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:51:01.178548 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:01.178556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.180024 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.180027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:01.181211 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:01.181222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.182334 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.182340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:01.182479 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.182556 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:51:01.199164 (+ 16608us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 06:51:01.199529 (+ 365us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.39.126:37099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":376,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.39.126:37099: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45895/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 06:51:01.202656 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23761
I20260619 06:51:01.203960 23822 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:01.299095 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23761
2026-06-19T06:51:01Z chronyd exiting
I20260619 06:51:01.349525 18589 test_util.cc:175] -----------------------------------------------
I20260619 06:51:01.349700 18589 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851818886250-18589-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (755 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 06:51:01.353689 18589 mini_oidc.cc:152] Starting JWKS server
I20260619 06:51:01.355528 18589 webserver.cc:460] Webserver started at https://127.0.0.1:38871/ using document root <none> and password file <none>
I20260619 06:51:01.355782 18589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:51:01.356439 18589 webserver.cc:460] Webserver started at http://127.0.0.1:37965/ using document root <none> and password file <none>
2026-06-19T06:51:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:01Z Disabled control of system clock
I20260619 06:51:01.383591 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42877
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:33041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:42877
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38871/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/testchainca.cert with env {}
W20260619 06:51:01.713804 23864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:01.714130 23864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:01.714221 23864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:01.721817 23864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:01.721948 23864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:01.721995 23864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:01.722048 23864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:01.722095 23864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:51:01.722152 23864 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:51:01.722193 23864 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38871/jwks.json
I20260619 06:51:01.732602 23864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:33041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:42877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38871/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:01.733999 23864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:01.735304 23864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:01.747476 23870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:01.747871 23872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:01.747941 23869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:01.750694 23864 server_base.cc:1034] running on GCE node
I20260619 06:51:01.751503 23864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:01.752961 23864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:01.754190 23864 hybrid_clock.cc:648] HybridClock initialized: now 1781851861754162 us; error 56 us; skew 500 ppm
I20260619 06:51:01.754559 23864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:01.756814 23864 webserver.cc:460] Webserver started at http://127.18.39.126:44969/ using document root <none> and password file <none>
I20260619 06:51:01.757586 23864 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:01.757721 23864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:01.758049 23864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:01.760901 23864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "b77ceb36b9974b239746b3ec02df6688"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "0bfb810ab8d92e0b6096ffc8c7bbce6d"
server_key_iv: "b2415a9218d64acea998b32b1635f302"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.761767 23864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "b77ceb36b9974b239746b3ec02df6688"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "0bfb810ab8d92e0b6096ffc8c7bbce6d"
server_key_iv: "b2415a9218d64acea998b32b1635f302"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.766983 23864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:51:01.771154 23878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:01.772521 23864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:51:01.772719 23864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "b77ceb36b9974b239746b3ec02df6688"
format_stamp: "Formatted at 2026-06-19 06:51:01 on dist-test-slave-1h5s"
server_key: "0bfb810ab8d92e0b6096ffc8c7bbce6d"
server_key_iv: "b2415a9218d64acea998b32b1635f302"
server_key_version: "encryptionkey@0"
I20260619 06:51:01.772953 23864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:01.792250 23864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:01.793139 23864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:01.793496 23864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:01.812057 23864 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:42877
I20260619 06:51:01.812068 23929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:42877 every 8 connection(s)
I20260619 06:51:01.813448 23864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:01.817188 23930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:01.823832 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23864
I20260619 06:51:01.824046 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:01.824842 18589 external_mini_cluster.cc:1428] Setting key 21d1ab2092f304214abcd5e2ed91e447
I20260619 06:51:01.828958 23930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688: Bootstrap starting.
I20260619 06:51:01.832105 23930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:01.833086 23930 log.cc:824] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:01.835968 23930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688: No bootstrap required, opened a new log
I20260619 06:51:01.838438 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:01.826725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.18.39.126:42877)
0619 06:51:01.827812 (+ 1087us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:01.827831 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:01.827885 (+ 54us) server_negotiation.cc:407] Connection header received
0619 06:51:01.829851 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:01.829873 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:01.830322 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:01.830694 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:01.831245 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.832553 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.833902 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.834563 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.834743 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.836101 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:01.836113 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:01.836133 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:01.836675 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:01.836748 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:01.836768 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:01.836895 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:01.837121 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 06:51:01.837477 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":347,"threads_started":1}
I20260619 06:51:01.842093 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:51:01.843715 23930 raft_consensus.cc:348] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } }
I20260619 06:51:01.844135 23930 raft_consensus.cc:374] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:01.844308 23930 raft_consensus.cc:729] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b77ceb36b9974b239746b3ec02df6688, State: Initialized, Role: FOLLOWER
I20260619 06:51:01.844905 23930 consensus_queue.cc:260] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } }
I20260619 06:51:01.845147 23930 raft_consensus.cc:388] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:01.845299 23930 raft_consensus.cc:482] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:01.845515 23930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:01.847630 23930 raft_consensus.cc:504] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } }
I20260619 06:51:01.848208 23930 leader_election.cc:302] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b77ceb36b9974b239746b3ec02df6688; no voters:
I20260619 06:51:01.848822 23930 leader_election.cc:288] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:01.848955 23935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:01.849699 23935 raft_consensus.cc:686] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [term 1 LEADER]: Becoming Leader. State: Replica: b77ceb36b9974b239746b3ec02df6688, State: Running, Role: LEADER
I20260619 06:51:01.850263 23935 consensus_queue.cc:237] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } }
I20260619 06:51:01.850955 23930 sys_catalog.cc:564] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:01.852809 23936 sys_catalog.cc:455] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b77ceb36b9974b239746b3ec02df6688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } } }
I20260619 06:51:01.853128 23936 sys_catalog.cc:458] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:01.853570 23937 sys_catalog.cc:455] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b77ceb36b9974b239746b3ec02df6688. Latest consensus state: current_term: 1 leader_uuid: "b77ceb36b9974b239746b3ec02df6688" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b77ceb36b9974b239746b3ec02df6688" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42877 } } }
I20260619 06:51:01.856048 23937 sys_catalog.cc:458] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:01.857280 23941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:01.862387 23941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:01.871199 23941 catalog_manager.cc:1347] Generated new cluster ID: 2f93da09df874630bc00151752f29d90
I20260619 06:51:01.871335 23941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:01.882779 23941 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:01.884086 23941 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:01.894354 23941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b77ceb36b9974b239746b3ec02df6688: Generated new TSK 0
I20260619 06:51:01.895241 23941 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 06:51:01.929242 23960 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 06:51:01.929430 23960 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:51:01.930676 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:01.909953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.18.39.126:42877)
0619 06:51:01.910118 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:01.910123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:01.910564 (+ 441us) server_negotiation.cc:407] Connection header received
0619 06:51:01.910771 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:01.910778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:01.910883 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:01.911062 (+ 179us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:51:01.911559 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.912703 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.914229 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.914924 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.915067 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.915636 (+ 569us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:51:01.929963 (+ 14327us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 06:51:01.930371 (+ 408us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 06:51:01.930809 23933 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 06:51:01.930821 23959 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:01.909834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42877 (local address 127.0.0.1:36738)
0619 06:51:01.910457 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:01.910473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:01.910597 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:01.911076 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:01.911079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:01.911088 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:51:01.911388 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:01.911397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.912861 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.912864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:01.914080 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:01.914093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:01.915127 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:01.915132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:01.915290 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:01.915343 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:51:01.930234 (+ 14891us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 06:51:01.930480 (+ 246us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.39.126:42877: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":340,"threads_started":1}
I20260619 06:51:01.931917 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23864
I20260619 06:51:01.933207 23925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:02.025357 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23864
2026-06-19T06:51:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (726 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 06:51:02.080051 18589 mini_oidc.cc:152] Starting JWKS server
I20260619 06:51:02.082021 18589 webserver.cc:460] Webserver started at https://127.0.0.1:34359/ using document root <none> and password file <none>
I20260619 06:51:02.082288 18589 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:51:02.082926 18589 webserver.cc:460] Webserver started at http://127.0.0.1:35161/ using document root <none> and password file <none>
2026-06-19T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:02Z Disabled control of system clock
I20260619 06:51:02.109920 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:33083
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:40603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:33083
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34359/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 06:51:02.436977 23967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:02.437275 23967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:02.437448 23967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:02.444733 23967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:02.444854 23967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:02.444892 23967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:02.444942 23967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:02.444986 23967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:51:02.445039 23967 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:51:02.445078 23967 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34359/jwks.json
I20260619 06:51:02.454561 23967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:33083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:33083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34359/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:02.455806 23967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:02.457540 23967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:02.469237 23975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:02.469367 23972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:02.469312 23973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:02.470024 23967 server_base.cc:1034] running on GCE node
I20260619 06:51:02.470844 23967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:02.472231 23967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:02.473449 23967 hybrid_clock.cc:648] HybridClock initialized: now 1781851862473397 us; error 56 us; skew 500 ppm
I20260619 06:51:02.473793 23967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:02.475984 23967 webserver.cc:460] Webserver started at http://127.18.39.126:35773/ using document root <none> and password file <none>
I20260619 06:51:02.476665 23967 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:02.476794 23967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:02.477113 23967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:02.479962 23967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "110c0239aaf54ca39cee15d369e18e76"
format_stamp: "Formatted at 2026-06-19 06:51:02 on dist-test-slave-1h5s"
server_key: "0bd841e61be2297501aee8f0e6c22f4b"
server_key_iv: "8e4a9ffc7470af10abe50fe6b74bb89e"
server_key_version: "encryptionkey@0"
I20260619 06:51:02.480731 23967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "110c0239aaf54ca39cee15d369e18e76"
format_stamp: "Formatted at 2026-06-19 06:51:02 on dist-test-slave-1h5s"
server_key: "0bd841e61be2297501aee8f0e6c22f4b"
server_key_iv: "8e4a9ffc7470af10abe50fe6b74bb89e"
server_key_version: "encryptionkey@0"
I20260619 06:51:02.485745 23967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 06:51:02.489090 23981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:02.490526 23967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:02.490710 23967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "110c0239aaf54ca39cee15d369e18e76"
format_stamp: "Formatted at 2026-06-19 06:51:02 on dist-test-slave-1h5s"
server_key: "0bd841e61be2297501aee8f0e6c22f4b"
server_key_iv: "8e4a9ffc7470af10abe50fe6b74bb89e"
server_key_version: "encryptionkey@0"
I20260619 06:51:02.490932 23967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:02.511053 23967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:02.511951 23967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:02.512269 23967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:02.531445 23967 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:33083
I20260619 06:51:02.531462 24032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:33083 every 8 connection(s)
I20260619 06:51:02.532851 23967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:02.536845 24033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:02.539646 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 23967
I20260619 06:51:02.539862 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:02.540647 18589 external_mini_cluster.cc:1428] Setting key 21f26bcc31c8035f2b84c2dacce80561
I20260619 06:51:02.556983 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76: Bootstrap starting.
I20260619 06:51:02.559446 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:02.544483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.18.39.126:33083)
0619 06:51:02.546905 (+ 2422us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:02.546921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:02.546976 (+ 55us) server_negotiation.cc:407] Connection header received
0619 06:51:02.550364 (+ 3388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:02.550387 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:02.550878 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:02.551338 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:02.551995 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.553673 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.554997 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.555674 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.555860 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:02.557071 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:02.557078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:02.557115 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:02.557757 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:02.557836 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:02.557856 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:02.557988 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:02.558195 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 06:51:02.558533 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1463,"thread_start_us":1218,"threads_started":1}
I20260619 06:51:02.561667 24033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:02.563095 24033 log.cc:824] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:02.563329 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:51:02.566957 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76: No bootstrap required, opened a new log
I20260619 06:51:02.581480 24033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } }
I20260619 06:51:02.582158 24033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:02.582325 24033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 110c0239aaf54ca39cee15d369e18e76, State: Initialized, Role: FOLLOWER
I20260619 06:51:02.583163 24033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } }
I20260619 06:51:02.583499 24033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:02.583667 24033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:02.583881 24033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:02.587962 24033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } }
I20260619 06:51:02.588675 24033 leader_election.cc:302] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 110c0239aaf54ca39cee15d369e18e76; no voters:
I20260619 06:51:02.589603 24033 leader_election.cc:288] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:02.589725 24038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:02.592253 24033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:02.593710 24038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [term 1 LEADER]: Becoming Leader. State: Replica: 110c0239aaf54ca39cee15d369e18e76, State: Running, Role: LEADER
I20260619 06:51:02.594336 24038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } }
I20260619 06:51:02.596889 24040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 110c0239aaf54ca39cee15d369e18e76. Latest consensus state: current_term: 1 leader_uuid: "110c0239aaf54ca39cee15d369e18e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } } }
I20260619 06:51:02.597239 24040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:02.597872 24039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "110c0239aaf54ca39cee15d369e18e76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "110c0239aaf54ca39cee15d369e18e76" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33083 } } }
I20260619 06:51:02.598161 24039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:02.601712 24043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:02.609640 24043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:02.623510 24043 catalog_manager.cc:1347] Generated new cluster ID: 25473124671640f3b7a9bf3cc4a13592
I20260619 06:51:02.623644 24043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:02.666070 24043 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:02.668442 24043 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:02.684891 24043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 110c0239aaf54ca39cee15d369e18e76: Generated new TSK 0
I20260619 06:51:02.686149 24043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:51:02.720006 24036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:02.704520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.18.39.126:33083)
0619 06:51:02.704745 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:02.704753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:02.710162 (+ 5409us) server_negotiation.cc:407] Connection header received
0619 06:51:02.710529 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:02.710535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:02.710640 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:02.710814 (+ 174us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:51:02.711491 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.712694 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.715200 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.715922 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.716073 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:02.716816 (+ 743us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:51:02.719277 (+ 2461us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34359/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 06:51:02.719623 (+ 346us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34359/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":74}
W20260619 06:51:02.720144 24036 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37250: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34359/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 06:51:02.721149 24062 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:02.704355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:33083 (local address 127.0.0.1:37250)
0619 06:51:02.709984 (+ 5629us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:02.710013 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:02.710257 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:02.710861 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:02.710870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:02.710889 (+ 19us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:51:02.711308 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:02.711324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.712885 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.712894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:02.715013 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:02.715034 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:02.716181 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:02.716190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:02.716435 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:02.716488 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:51:02.720404 (+ 3916us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34359/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 06:51:02.720702 (+ 298us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.39.126:33083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34359/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":5425,"thread_start_us":5226,"threads_started":1}
I20260619 06:51:02.722319 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 23967
I20260619 06:51:02.723991 24028 generic_service.cc:196] Checking for leaks (request via RPC)
E20260619 06:51:02.737632 24063 webserver.cc:533] Webserver: sslize failed:
E20260619 06:51:02.737777 24063 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:51:02.847355 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 23967
2026-06-19T06:51:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (826 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T06:51:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:02Z Disabled control of system clock
I20260619 06:51:02.933712 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37431
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:37431
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/testchainca.cert with env {}
W20260619 06:51:03.266294 24068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:03.266613 24068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:03.266785 24068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:03.277675 24068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:03.277798 24068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:03.277889 24068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:03.277977 24068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:03.278051 24068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:51:03.278142 24068 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:51:03.278218 24068 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 06:51:03.288471 24068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:37431
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:03.289945 24068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:03.291270 24068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:03.303887 24076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:03.303923 24074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:03.304224 24073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:03.304872 24068 server_base.cc:1034] running on GCE node
I20260619 06:51:03.305728 24068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:03.307152 24068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:03.308351 24068 hybrid_clock.cc:648] HybridClock initialized: now 1781851863308290 us; error 53 us; skew 500 ppm
I20260619 06:51:03.308734 24068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:03.311254 24068 webserver.cc:460] Webserver started at http://127.18.39.126:45023/ using document root <none> and password file <none>
I20260619 06:51:03.312000 24068 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:03.312119 24068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:03.312446 24068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:03.316040 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "b170129966374f05b86ef667e522519d"
format_stamp: "Formatted at 2026-06-19 06:51:03 on dist-test-slave-1h5s"
server_key: "8d361b473ba79afa872dc703217d9f2e"
server_key_iv: "19af148a7a4537181af70d3f4ef016ce"
server_key_version: "encryptionkey@0"
I20260619 06:51:03.316926 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "b170129966374f05b86ef667e522519d"
format_stamp: "Formatted at 2026-06-19 06:51:03 on dist-test-slave-1h5s"
server_key: "8d361b473ba79afa872dc703217d9f2e"
server_key_iv: "19af148a7a4537181af70d3f4ef016ce"
server_key_version: "encryptionkey@0"
I20260619 06:51:03.322890 24068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260619 06:51:03.326625 24082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:03.328258 24068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:51:03.328527 24068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "b170129966374f05b86ef667e522519d"
format_stamp: "Formatted at 2026-06-19 06:51:03 on dist-test-slave-1h5s"
server_key: "8d361b473ba79afa872dc703217d9f2e"
server_key_iv: "19af148a7a4537181af70d3f4ef016ce"
server_key_version: "encryptionkey@0"
I20260619 06:51:03.328768 24068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:03.344056 24068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:03.344980 24068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:03.345302 24068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:03.364269 24068 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:37431
I20260619 06:51:03.364292 24133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:37431 every 8 connection(s)
I20260619 06:51:03.365693 24068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:03.369601 24134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:03.371943 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24068
I20260619 06:51:03.372152 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:03.372810 18589 external_mini_cluster.cc:1428] Setting key a71c316d118db0d0ad07ed290b57b504
I20260619 06:51:03.382997 24134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d: Bootstrap starting.
I20260619 06:51:03.387247 24134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:03.388545 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:03.374857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38588 (local address 127.18.39.126:37431)
0619 06:51:03.375936 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:03.375951 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:03.376000 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:51:03.378346 (+ 2346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:03.378376 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:03.379029 (+ 653us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:03.379488 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:03.380054 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.382027 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.383362 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.384350 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.384540 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:03.385689 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:03.385699 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:03.385716 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:03.386432 (+ 716us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:03.386549 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:03.386575 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:03.386685 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:03.386868 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 06:51:03.387221 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":313,"threads_started":1}
I20260619 06:51:03.388625 24134 log.cc:824] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:03.391922 24134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d: No bootstrap required, opened a new log
I20260619 06:51:03.392712 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:51:03.400002 24134 raft_consensus.cc:348] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } }
I20260619 06:51:03.400431 24134 raft_consensus.cc:374] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:03.400539 24134 raft_consensus.cc:729] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b170129966374f05b86ef667e522519d, State: Initialized, Role: FOLLOWER
I20260619 06:51:03.401144 24134 consensus_queue.cc:260] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } }
I20260619 06:51:03.401402 24134 raft_consensus.cc:388] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:03.401557 24134 raft_consensus.cc:482] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:03.401732 24134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:03.403896 24134 raft_consensus.cc:504] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } }
I20260619 06:51:03.404409 24134 leader_election.cc:302] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b170129966374f05b86ef667e522519d; no voters:
I20260619 06:51:03.405006 24134 leader_election.cc:288] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:03.405175 24139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:03.405949 24139 raft_consensus.cc:686] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [term 1 LEADER]: Becoming Leader. State: Replica: b170129966374f05b86ef667e522519d, State: Running, Role: LEADER
I20260619 06:51:03.406610 24139 consensus_queue.cc:237] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } }
I20260619 06:51:03.407166 24134 sys_catalog.cc:564] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:03.409835 24140 sys_catalog.cc:455] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b170129966374f05b86ef667e522519d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } } }
I20260619 06:51:03.410215 24140 sys_catalog.cc:458] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:03.411787 24147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:03.411732 24141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b170129966374f05b86ef667e522519d. Latest consensus state: current_term: 1 leader_uuid: "b170129966374f05b86ef667e522519d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b170129966374f05b86ef667e522519d" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37431 } } }
I20260619 06:51:03.412058 24141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:03.420711 24147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:03.430014 24147 catalog_manager.cc:1347] Generated new cluster ID: 81772544da204a9891f0904c435f324e
I20260619 06:51:03.430168 24147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:03.445849 24147 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:03.447736 24147 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:03.467173 24147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b170129966374f05b86ef667e522519d: Generated new TSK 0
I20260619 06:51:03.468334 24147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:51:03.516789 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:03.491413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.18.39.126:37431)
0619 06:51:03.491588 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:03.491593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:03.491957 (+ 364us) server_negotiation.cc:407] Connection header received
0619 06:51:03.492183 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:03.492189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:03.492285 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:03.492467 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:51:03.492916 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.494101 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.495608 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.496295 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.496425 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:03.496972 (+ 547us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:51:03.516034 (+ 19062us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 06:51:03.516454 (+ 420us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38590: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 06:51:03.516952 24137 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38590: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 06:51:03.517069 24163 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:03.491226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:37431 (local address 127.0.0.1:38590)
0619 06:51:03.491843 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:03.491860 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:03.491991 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:03.492480 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:03.492484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:03.492493 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:51:03.492736 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:03.492743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.494297 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.494302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:03.495453 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:03.495466 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:03.496458 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:03.496462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:03.496604 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:03.496663 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:51:03.516320 (+ 19657us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 06:51:03.516617 (+ 297us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.39.126:37431: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":323,"threads_started":1}
I20260619 06:51:03.518625 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24068
I20260619 06:51:03.519814 24129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:03.615974 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24068
2026-06-19T06:51:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (744 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (413 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (428 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:04 dist-test-slave-1h5s krb5kdc[24178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:04 dist-test-slave-1h5s krb5kdc[24178](info): set up 2 sockets
Jun 19 06:51:04 dist-test-slave-1h5s krb5kdc[24178](info): commencing operation
krb5kdc: starting...
W20260619 06:51:05.553853 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:05 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:51:05.766170 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42431
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:39243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:42431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:06.091897 24191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:06.092262 24191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:06.092417 24191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:06.100072 24191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:06.100267 24191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:06.100392 24191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:06.100502 24191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:06.100577 24191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:06.110889 24191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:42431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:06.112339 24191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:06.113683 24191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:06.125931 24197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:06.125958 24199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:06.125731 24196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:06.126945 24191 server_base.cc:1034] running on GCE node
I20260619 06:51:06.127770 24191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:06.129317 24191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:06.130630 24191 hybrid_clock.cc:648] HybridClock initialized: now 1781851866130572 us; error 52 us; skew 500 ppm
I20260619 06:51:06.131003 24191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:06 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:06.139967 24191 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:51:06.141729 24191 webserver.cc:460] Webserver started at http://127.18.39.126:39959/ using document root <none> and password file <none>
I20260619 06:51:06.142499 24191 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:06.142656 24191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:06.142997 24191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:06.146127 24191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "61922bb276f74dde85e720812b6b0ad4"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "d178b866148cf45de2349064e69d8005"
server_key_iv: "054ba482042d2cbff127d64a57c79b72"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.146968 24191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "61922bb276f74dde85e720812b6b0ad4"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "d178b866148cf45de2349064e69d8005"
server_key_iv: "054ba482042d2cbff127d64a57c79b72"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.152359 24191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:51:06.156032 24206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:06.157488 24191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:06.157629 24191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "61922bb276f74dde85e720812b6b0ad4"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "d178b866148cf45de2349064e69d8005"
server_key_iv: "054ba482042d2cbff127d64a57c79b72"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.157814 24191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:06.176740 24191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:06.180997 24191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:06.181236 24191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:06.199844 24191 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:42431
I20260619 06:51:06.199868 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:42431 every 8 connection(s)
I20260619 06:51:06.201267 24191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:06.204885 24258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:06.207633 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24191
I20260619 06:51:06.207852 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:06.208487 18589 external_mini_cluster.cc:1428] Setting key fb52924c3ea6de77c81eba4eccb7aa2f
I20260619 06:51:06.217175 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4: Bootstrap starting.
I20260619 06:51:06.221135 24258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:06.222559 24258 log.cc:824] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:51:06 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:06.225651 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4: No bootstrap required, opened a new log
I20260619 06:51:06.231827 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:06.210515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.18.39.126:42431)
0619 06:51:06.211292 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:06.211304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:06.211343 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:51:06.213003 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:06.213025 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:06.213505 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:06.213834 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:06.215557 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.216830 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.218623 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.219364 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.219508 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:06.224345 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:06.224354 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:06.224370 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:06.224406 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:06.227631 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:06.228366 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:06.228376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:06.228383 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:06.228557 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:06.228943 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:06.228953 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:06.228961 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:06.229572 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:06.229918 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:06.230347 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 06:51:06.230776 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":341,"threads_started":1}
I20260619 06:51:06.233785 24258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } }
I20260619 06:51:06.234154 24258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:06.234266 24258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61922bb276f74dde85e720812b6b0ad4, State: Initialized, Role: FOLLOWER
I20260619 06:51:06.234954 24258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } }
I20260619 06:51:06.235217 24258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:06.235368 24258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:06.235538 24258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:06.237793 24258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } }
I20260619 06:51:06.238300 24258 leader_election.cc:302] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61922bb276f74dde85e720812b6b0ad4; no voters:
I20260619 06:51:06.238904 24258 leader_election.cc:288] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:06.239053 24263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:06.239781 24263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [term 1 LEADER]: Becoming Leader. State: Replica: 61922bb276f74dde85e720812b6b0ad4, State: Running, Role: LEADER
I20260619 06:51:06.240378 24263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } }
I20260619 06:51:06.241173 24258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:06.243827 24265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61922bb276f74dde85e720812b6b0ad4. Latest consensus state: current_term: 1 leader_uuid: "61922bb276f74dde85e720812b6b0ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } } }
I20260619 06:51:06.244289 24265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:06.245998 24271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:06.249727 24264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61922bb276f74dde85e720812b6b0ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61922bb276f74dde85e720812b6b0ad4" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42431 } } }
I20260619 06:51:06.250100 24264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:06.251912 24271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:06.261793 24271 catalog_manager.cc:1347] Generated new cluster ID: 383ce465eb5b49a194562796064d8482
I20260619 06:51:06.261924 24271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:06.275468 24271 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:06.277017 24271 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:06.283928 24271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61922bb276f74dde85e720812b6b0ad4: Generated new TSK 0
I20260619 06:51:06.285001 24271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:51:06.349838 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:39243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:06.685724 24284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:06.686082 24284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:06.686259 24284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:06.693785 24284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:06.693945 24284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:06.694118 24284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:06.703963 24284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:06.705410 24284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:06.706805 24284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:06.720453 24289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:06.720432 24290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:06.720772 24292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:06.721570 24284 server_base.cc:1034] running on GCE node
I20260619 06:51:06.722146 24284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:06.723024 24284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:06.724203 24284 hybrid_clock.cc:648] HybridClock initialized: now 1781851866724136 us; error 56 us; skew 500 ppm
I20260619 06:51:06.724531 24284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:06 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:06.727723 24284 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:51:06.729445 24284 webserver.cc:460] Webserver started at http://127.18.39.65:39255/ using document root <none> and password file <none>
I20260619 06:51:06.730188 24284 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:06.730351 24284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:06.730693 24284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:06.733958 24284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "c8a8d234bd20494d8ab7a7e127005762"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "f2cb9991410fa732e6f5aa4c63f44c55"
server_key_iv: "068c75482974a6c4a0b2d7852750853e"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.734818 24284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "c8a8d234bd20494d8ab7a7e127005762"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "f2cb9991410fa732e6f5aa4c63f44c55"
server_key_iv: "068c75482974a6c4a0b2d7852750853e"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.740554 24284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:51:06.744679 24299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:06.746204 24284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:06.746402 24284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "c8a8d234bd20494d8ab7a7e127005762"
format_stamp: "Formatted at 2026-06-19 06:51:06 on dist-test-slave-1h5s"
server_key: "f2cb9991410fa732e6f5aa4c63f44c55"
server_key_iv: "068c75482974a6c4a0b2d7852750853e"
server_key_version: "encryptionkey@0"
I20260619 06:51:06.746621 24284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:06.771813 24284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:06.776371 24284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:06.776692 24284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:06.777827 24284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:06.779269 24284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:06.779386 24284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:06.779546 24284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:06.779628 24284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:06.815258 24284 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42537
I20260619 06:51:06.815315 24411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42537 every 8 connection(s)
I20260619 06:51:06.816624 24284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:06.825591 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24284
I20260619 06:51:06.825786 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:06.826438 18589 external_mini_cluster.cc:1428] Setting key d8e1b3bb6b258d18ccdf806649de667f
Jun 19 06:51:06 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:06.838843 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:06.819198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:56551 (local address 127.18.39.126:42431)
0619 06:51:06.819667 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:06.819673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:06.820620 (+ 947us) server_negotiation.cc:407] Connection header received
0619 06:51:06.822526 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:06.822530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:06.822614 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:06.822731 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:06.824186 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.824933 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.826878 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.827438 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.827570 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:06.831773 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:06.831782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:06.831789 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:06.831834 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:06.835229 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:06.836015 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:06.836050 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:06.836058 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:06.836218 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:06.836673 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:06.836682 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:06.836689 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:06.837099 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:06.837261 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:06.838029 (+ 768us) server_negotiation.cc:299] Negotiation successful
0619 06:51:06.838299 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260619 06:51:06.839381 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:06.819749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42431 (local address 127.18.39.65:56551)
0619 06:51:06.820462 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:06.820505 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:06.822254 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:06.822962 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:06.822970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:06.823466 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:06.824028 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:06.824049 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.825081 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.825085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:06.826722 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:06.826734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:06.827672 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:06.827679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:06.827973 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:06.828893 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:06.828918 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:06.831530 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:06.835446 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:06.835453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:06.835466 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:06.835870 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:06.836365 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:06.836368 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:06.836371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:06.836553 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:06.837224 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:06.837230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:06.837635 (+ 405us) client_negotiation.cc:769] Sending connection context
0619 06:51:06.837859 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 06:51:06.838246 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":287,"threads_started":1}
I20260619 06:51:06.840974 24412 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42431
I20260619 06:51:06.841293 24412 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:06.842054 24412 heartbeater.cc:511] Master 127.18.39.126:42431 requested a full tablet report, sending...
I20260619 06:51:06.844275 24223 ts_manager.cc:194] Registered new tserver with Master: c8a8d234bd20494d8ab7a7e127005762 (127.18.39.65:42537)
I20260619 06:51:06.845774 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:56551
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:51:06.876315 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:39243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:07.200500 24418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:07.200862 24418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:07.201018 24418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:07.212672 24418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:07.212809 24418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:07.212977 24418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:07.226078 24418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:07.227281 24418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:07.228539 24418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:07.242643 24424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:07.242769 24426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:07.242682 24423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:07.248993 24418 server_base.cc:1034] running on GCE node
I20260619 06:51:07.249738 24418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:07.250878 24418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:07.252080 24418 hybrid_clock.cc:648] HybridClock initialized: now 1781851867252028 us; error 65 us; skew 500 ppm
I20260619 06:51:07.252424 24418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.255573 24418 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:51:07.256995 24418 webserver.cc:460] Webserver started at http://127.18.39.66:40087/ using document root <none> and password file <none>
I20260619 06:51:07.257756 24418 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:07.257900 24418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:07.258203 24418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:07.261061 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "dc0e84dfadc542fa9fd8cdcacb109bc1"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "5b4b2244e22308f70c62e305598ff87e"
server_key_iv: "3ef92e074931386f0cf20eb0c6257a51"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.261873 24418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "dc0e84dfadc542fa9fd8cdcacb109bc1"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "5b4b2244e22308f70c62e305598ff87e"
server_key_iv: "3ef92e074931386f0cf20eb0c6257a51"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.266825 24418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:51:07.270124 24433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.271441 24418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:07.271624 24418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "dc0e84dfadc542fa9fd8cdcacb109bc1"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "5b4b2244e22308f70c62e305598ff87e"
server_key_iv: "3ef92e074931386f0cf20eb0c6257a51"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.271840 24418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:07.297081 24418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:07.301254 24418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:07.301610 24418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:07.302614 24418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:07.304018 24418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:07.304123 24418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.304271 24418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:07.304342 24418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.338389 24418 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:34759
I20260619 06:51:07.338485 24545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:34759 every 8 connection(s)
I20260619 06:51:07.339807 24418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:07.347584 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24418
I20260619 06:51:07.347800 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:07.348342 18589 external_mini_cluster.cc:1428] Setting key 7161086ec80922dd2648c92f73a5d254
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:07.361074 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.342371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:37221 (local address 127.18.39.126:42431)
0619 06:51:07.342847 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.342854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.343924 (+ 1070us) server_negotiation.cc:407] Connection header received
0619 06:51:07.345850 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.345855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.345932 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.346038 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.348125 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.348891 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.350955 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.351469 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.351630 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.355246 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:07.355257 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:07.355261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:07.355292 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:07.358004 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.358739 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.358744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.358748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.358879 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.359315 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.359320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.359323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.359589 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:07.359738 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:07.360452 (+ 714us) server_negotiation.cc:299] Negotiation successful
0619 06:51:07.360676 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":235,"threads_started":1}
I20260619 06:51:07.362187 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.343019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42431 (local address 127.18.39.66:37221)
0619 06:51:07.343763 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:07.343805 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:07.345552 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:07.346411 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:07.346420 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:07.346969 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:07.347918 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:07.347947 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.349060 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.349065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:07.350735 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:07.350747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.351721 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.351728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:07.351957 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.352688 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:07.352712 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:07.355002 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:07.358144 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:07.358154 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:07.358174 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:07.358608 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:07.359003 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:07.359008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:07.359011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:07.359199 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:07.359720 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:07.359728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:07.360082 (+ 354us) client_negotiation.cc:769] Sending connection context
0619 06:51:07.360361 (+ 279us) client_negotiation.cc:240] Negotiation successful
0619 06:51:07.360902 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":290,"threads_started":1}
I20260619 06:51:07.363606 24546 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42431
I20260619 06:51:07.363914 24546 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:07.364621 24546 heartbeater.cc:511] Master 127.18.39.126:42431 requested a full tablet report, sending...
I20260619 06:51:07.366281 24223 ts_manager.cc:194] Registered new tserver with Master: dc0e84dfadc542fa9fd8cdcacb109bc1 (127.18.39.66:34759)
I20260619 06:51:07.367576 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:37221
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:51:07.396337 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:39243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:07.727720 24552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:07.728008 24552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:07.728188 24552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:07.735565 24552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:07.735684 24552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:07.735853 24552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:07.746203 24552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:39243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:07.747552 24552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:07.748900 24552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:07.763001 24557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:07.762959 24560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:07.763051 24558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:07.764120 24552 server_base.cc:1034] running on GCE node
I20260619 06:51:07.764788 24552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:07.765971 24552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:07.767254 24552 hybrid_clock.cc:648] HybridClock initialized: now 1781851867767202 us; error 47 us; skew 500 ppm
I20260619 06:51:07.767663 24552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.771166 24552 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:51:07.772782 24552 webserver.cc:460] Webserver started at http://127.18.39.67:46285/ using document root <none> and password file <none>
I20260619 06:51:07.773584 24552 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:07.773722 24552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:07.774045 24552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:07.777244 24552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "1e2b1e1418f94134b7e5b8929dc78fb8"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "8bbe5db7f943417a6c55f860d1005e51"
server_key_iv: "0506352e208b331fcee4be395ece5154"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.778111 24552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "1e2b1e1418f94134b7e5b8929dc78fb8"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "8bbe5db7f943417a6c55f860d1005e51"
server_key_iv: "0506352e208b331fcee4be395ece5154"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.783241 24552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:07.786760 24567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.788098 24552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:07.788275 24552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "1e2b1e1418f94134b7e5b8929dc78fb8"
format_stamp: "Formatted at 2026-06-19 06:51:07 on dist-test-slave-1h5s"
server_key: "8bbe5db7f943417a6c55f860d1005e51"
server_key_iv: "0506352e208b331fcee4be395ece5154"
server_key_version: "encryptionkey@0"
I20260619 06:51:07.788486 24552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:07.836429 24552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:07.841092 24552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:07.841532 24552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:07.842621 24552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:07.843982 24552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:07.844085 24552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.844245 24552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:07.844326 24552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:07.848726 24412 heartbeater.cc:503] Master 127.18.39.126:42431 was elected leader, sending a full tablet report...
I20260619 06:51:07.878302 24552 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:39673
I20260619 06:51:07.878393 24679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:39673 every 8 connection(s)
I20260619 06:51:07.879601 24552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:07.887667 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24552
I20260619 06:51:07.887850 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:07.888357 18589 external_mini_cluster.cc:1428] Setting key a194779dd3696b50467fd24afb2a747b
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:07.900594 24682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.882139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:57757 (local address 127.18.39.126:42431)
0619 06:51:07.882551 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.882556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.883534 (+ 978us) server_negotiation.cc:407] Connection header received
0619 06:51:07.885385 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.885391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.885495 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.885615 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.887114 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.888294 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.890764 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.891233 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.891348 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.895139 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:07.895144 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:07.895147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:07.895180 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:07.897905 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.898646 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.898649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.898652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.898761 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.899122 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.899124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.899126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.899371 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:07.899484 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:07.900157 (+ 673us) server_negotiation.cc:299] Negotiation successful
0619 06:51:07.900335 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":213,"threads_started":1}
I20260619 06:51:07.901643 24683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.882693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42431 (local address 127.18.39.67:57757)
0619 06:51:07.883366 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:07.883410 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:07.885080 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:07.885858 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:07.885867 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:07.886372 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:07.886940 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:07.886957 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.888540 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.888544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:07.890587 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:07.890605 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.891490 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.891498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:07.891845 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.892565 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:07.892587 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:07.894935 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:07.898040 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:07.898048 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:07.898063 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:07.898532 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:07.898852 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:07.898855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:07.898857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:07.899032 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:07.899476 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:07.899481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:07.899798 (+ 317us) client_negotiation.cc:769] Sending connection context
0619 06:51:07.900032 (+ 234us) client_negotiation.cc:240] Negotiation successful
0619 06:51:07.900427 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":284,"threads_started":1}
I20260619 06:51:07.903221 24680 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42431
I20260619 06:51:07.903542 24680 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:07.904424 24680 heartbeater.cc:511] Master 127.18.39.126:42431 requested a full tablet report, sending...
I20260619 06:51:07.905838 24223 ts_manager.cc:194] Registered new tserver with Master: 1e2b1e1418f94134b7e5b8929dc78fb8 (127.18.39.67:39673)
I20260619 06:51:07.906630 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:57757
I20260619 06:51:07.917060 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/test-admin.keytab.
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.943152 18589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 06:51:07.948413 18589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc' of type FILE
W20260619 06:51:07.948503 24688 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:07.946007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.18.39.65:42537)
0619 06:51:07.946688 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.946696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.946744 (+ 48us) server_negotiation.cc:407] Connection header received
0619 06:51:07.946815 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.946822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.947041 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.947186 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.948033 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45662: BlockingRecv error: recv got EOF from 127.0.0.1:45662 (error 108)
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":375,"threads_started":1}
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.950155 18589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.951645 18589 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:51:07.968142 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.953261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.18.39.65:42537)
0619 06:51:07.953507 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.953518 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.953818 (+ 300us) server_negotiation.cc:407] Connection header received
0619 06:51:07.953936 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.953939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.954000 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.954088 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.955000 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.956076 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.957262 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.957932 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.958038 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.961647 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:07.961659 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:07.961674 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:07.961714 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:07.964816 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.965666 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.965674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.965676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.965813 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.966241 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.966252 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.966254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.966761 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:07.967110 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:07.967529 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 06:51:07.967819 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 06:51:07.973984 24688 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:07.972367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.18.39.65:42537)
0619 06:51:07.972576 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.972581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.972655 (+ 74us) server_negotiation.cc:407] Connection header received
0619 06:51:07.972783 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.972787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.972860 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.972940 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.973703 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45680: BlockingRecv error: recv got EOF from 127.0.0.1:45680 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 06:51:07.974088 18589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc' of type FILE
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.975785 18589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.977262 18589 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:51:07.991315 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:07.978659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45696 (local address 127.18.39.65:42537)
0619 06:51:07.978897 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.978903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.979144 (+ 241us) server_negotiation.cc:407] Connection header received
0619 06:51:07.979321 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.979325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.979384 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.979465 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.980472 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.981149 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.982424 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:07.982931 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:07.983085 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:07.986633 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:07.986638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:07.986642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:07.986673 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:07.989084 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.989733 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.989737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.989739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.989853 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:07.990159 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:07.990162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:07.990164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:07.990407 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:07.990595 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:07.990833 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 06:51:07.991015 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260619 06:51:07.995523 24688 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:07.993956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.18.39.65:42537)
0619 06:51:07.994176 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:07.994182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:07.994334 (+ 152us) server_negotiation.cc:407] Connection header received
0619 06:51:07.994455 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:07.994458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:07.994527 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:07.994604 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:07.995289 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45698: BlockingRecv error: recv got EOF from 127.0.0.1:45698 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260619 06:51:07.995823 18589 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851818886250-18589-0/krb5kdc/krb5cc' of type FILE
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.997252 18589 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:51:07 dist-test-slave-1h5s krb5kdc[24178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:07.998639 18589 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:51:08.013666 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:08.000193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45702 (local address 127.18.39.65:42537)
0619 06:51:08.000401 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:08.000409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:08.000611 (+ 202us) server_negotiation.cc:407] Connection header received
0619 06:51:08.000834 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:08.000839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:08.000898 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:08.001005 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:08.002037 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.002751 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.003982 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.004425 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.004565 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:08.008236 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:08.008241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:08.008244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:08.008282 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:08.010864 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.011594 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.011606 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.011609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.011754 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.012223 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.012229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.012233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.012523 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:08.012822 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:08.013103 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 06:51:08.013314 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:51:08.015259 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24284
I20260619 06:51:08.027736 24688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:08.015981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.18.39.65:42537)
0619 06:51:08.016157 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:08.016165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:08.016451 (+ 286us) server_negotiation.cc:407] Connection header received
0619 06:51:08.016647 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:08.016652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:08.016745 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:08.016862 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:08.017902 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.018578 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.019777 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.020211 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.020345 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:08.022539 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:08.022546 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:08.022550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:08.022589 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:08.024991 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.025748 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.025753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.025757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.025892 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.026291 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.026300 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.026302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.026568 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:08.026849 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:08.027208 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 06:51:08.027425 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:51:08.028296 24407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:08.111624 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24284
I20260619 06:51:08.127579 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24418
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:51:08.143862 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:08.128476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.18.39.66:34759)
0619 06:51:08.129065 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:08.129073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:08.129097 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:51:08.129158 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:08.129164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:08.129436 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:08.129607 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:08.130603 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.131746 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.132999 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.133749 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.133894 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:08.137856 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:08.137868 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:08.137883 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:08.137917 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:08.140776 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.141608 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.141616 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.141618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.141739 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.142137 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.142147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.142149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.142505 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:08.142852 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:08.143186 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 06:51:08.143442 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":309,"threads_started":1}
I20260619 06:51:08.145185 24541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:08.227984 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24418
I20260619 06:51:08.243512 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24552
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:51:08.259297 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:08.244431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.18.39.67:39673)
0619 06:51:08.245071 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:08.245077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:08.245107 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:51:08.245165 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:08.245171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:08.245475 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:08.245614 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:08.246585 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.247630 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.248899 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:08.249671 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:08.249771 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:08.253962 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:08.253974 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:08.253990 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:08.254028 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:08.256818 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.257504 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.257511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.257513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.257624 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:08.257955 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:08.257958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:08.257960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:08.258283 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:08.258526 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:08.258726 (+ 200us) server_negotiation.cc:299] Negotiation successful
0619 06:51:08.258980 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":321,"threads_started":1}
I20260619 06:51:08.260756 24675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:08.349943 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24552
I20260619 06:51:08.370877 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24191
I20260619 06:51:08.372089 24253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:08.475371 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24191
2026-06-19T06:51:08Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4022 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24708](info): set up 2 sockets
Jun 19 06:51:08 dist-test-slave-1h5s krb5kdc[24708](info): commencing operation
krb5kdc: starting...
W20260619 06:51:09.574170 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:09 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab.
Entry for principal oryx/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab.
I20260619 06:51:09.789507 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40475
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab
--principal=oryx/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:10.121136 24721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:10.121542 24721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:10.121690 24721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:10.129159 24721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:10.129266 24721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:10.129376 24721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:10.129483 24721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:10.129542 24721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:10.139704 24721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab
--principal=oryx/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:10.141083 24721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:10.142464 24721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:10.154186 24729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:10.154273 24726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:10.154248 24727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:10.155259 24721 server_base.cc:1034] running on GCE node
I20260619 06:51:10.156095 24721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:10.157516 24721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:10.158747 24721 hybrid_clock.cc:648] HybridClock initialized: now 1781851870158699 us; error 57 us; skew 500 ppm
I20260619 06:51:10.159098 24721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:10 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851870, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:10.162315 24721 init.cc:377] Logged in from keytab as oryx/127.18.39.126@KRBTEST.COM (short username oryx)
I20260619 06:51:10.163925 24721 webserver.cc:460] Webserver started at http://127.18.39.126:35245/ using document root <none> and password file <none>
I20260619 06:51:10.164636 24721 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:10.164783 24721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:10.165097 24721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:10.168166 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "c7bcafeefff94aedad77a5a8a483df88"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "2a061472ca0ceb2be4815ef7ef2d47f8"
server_key_iv: "26e76c023f40f69723058dc8fcb904d3"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.168956 24721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "c7bcafeefff94aedad77a5a8a483df88"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "2a061472ca0ceb2be4815ef7ef2d47f8"
server_key_iv: "26e76c023f40f69723058dc8fcb904d3"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.174403 24721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:51:10.178013 24736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:10.179463 24721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:10.179662 24721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "c7bcafeefff94aedad77a5a8a483df88"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "2a061472ca0ceb2be4815ef7ef2d47f8"
server_key_iv: "26e76c023f40f69723058dc8fcb904d3"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.179879 24721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:10.210912 24721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:10.215523 24721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:10.215838 24721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:10.234419 24721 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:40475
I20260619 06:51:10.234441 24787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:40475 every 8 connection(s)
I20260619 06:51:10.235884 24721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:10.239825 24788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:10.240937 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24721
I20260619 06:51:10.241168 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:10.242164 18589 external_mini_cluster.cc:1428] Setting key 002c3e58e026c101ceab74ddc5076dd2
I20260619 06:51:10.255149 24788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Bootstrap starting.
Jun 19 06:51:10 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:10.259153 24788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:10.260594 24788 log.cc:824] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:10.264117 24788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: No bootstrap required, opened a new log
I20260619 06:51:10.267102 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:10.244438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.18.39.126:40475)
0619 06:51:10.245259 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:10.245274 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:10.245376 (+ 102us) server_negotiation.cc:407] Connection header received
0619 06:51:10.246977 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:10.246999 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:10.247433 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:10.247776 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:10.249565 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.250868 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.252772 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.253641 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.253789 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:10.258604 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:10.258616 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:10.258635 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:10.258684 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:10.262496 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:10.263204 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:10.263215 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:10.263226 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:10.263434 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:10.263783 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:10.263788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:10.263790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:10.264497 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:10.264843 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:10.265209 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 06:51:10.265667 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":389,"threads_started":1}
I20260619 06:51:10.273326 24788 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } }
I20260619 06:51:10.273831 24788 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:10.273957 24788 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bcafeefff94aedad77a5a8a483df88, State: Initialized, Role: FOLLOWER
I20260619 06:51:10.274691 24788 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } }
I20260619 06:51:10.274993 24788 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:10.275127 24788 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:10.275311 24788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:10.277592 24788 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } }
I20260619 06:51:10.278163 24788 leader_election.cc:302] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bcafeefff94aedad77a5a8a483df88; no voters:
I20260619 06:51:10.278792 24788 leader_election.cc:288] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:10.278954 24793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:10.280117 24793 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 LEADER]: Becoming Leader. State: Replica: c7bcafeefff94aedad77a5a8a483df88, State: Running, Role: LEADER
I20260619 06:51:10.280759 24793 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } }
I20260619 06:51:10.281203 24788 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:10.284196 24794 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } }
I20260619 06:51:10.284862 24794 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:10.284206 24795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcafeefff94aedad77a5a8a483df88. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } }
I20260619 06:51:10.285869 24795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:10.291152 24800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:10.297091 24800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:10.307348 24800 catalog_manager.cc:1347] Generated new cluster ID: bc687f4e85d147699224912d02287b09
I20260619 06:51:10.307494 24800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:10.316588 24800 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:10.318166 24800 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:10.330224 24800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Generated new TSK 0
I20260619 06:51:10.331383 24800 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
I20260619 06:51:10.405293 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:10.763669 24814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:10.763976 24814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:10.764145 24814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:10.771610 24814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:10.771733 24814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:10.771948 24814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:10.782286 24814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:10.783785 24814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:10.785260 24814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:10.798764 24819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:10.798722 24820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:10.798929 24822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:10.799801 24814 server_base.cc:1034] running on GCE node
I20260619 06:51:10.800547 24814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:10.801656 24814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:10.802884 24814 hybrid_clock.cc:648] HybridClock initialized: now 1781851870802836 us; error 71 us; skew 500 ppm
I20260619 06:51:10.803213 24814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:10 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851870, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:10.806358 24814 init.cc:377] Logged in from keytab as oryx/127.18.39.65@KRBTEST.COM (short username oryx)
I20260619 06:51:10.807775 24814 webserver.cc:460] Webserver started at http://127.18.39.65:41145/ using document root <none> and password file <none>
I20260619 06:51:10.808523 24814 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:10.808622 24814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:10.808881 24814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:10.811856 24814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "e467617eed594e9a9611d0a3b1bdd644"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "51a489ae2af23ab9f5b092d2313eaf60"
server_key_iv: "0c54060586f21780510ba49065c4e1b4"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.812657 24814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "e467617eed594e9a9611d0a3b1bdd644"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "51a489ae2af23ab9f5b092d2313eaf60"
server_key_iv: "0c54060586f21780510ba49065c4e1b4"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.817909 24814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260619 06:51:10.821470 24829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:10.822942 24814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:10.823082 24814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "e467617eed594e9a9611d0a3b1bdd644"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "51a489ae2af23ab9f5b092d2313eaf60"
server_key_iv: "0c54060586f21780510ba49065c4e1b4"
server_key_version: "encryptionkey@0"
I20260619 06:51:10.823266 24814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:10.838339 24814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:10.842718 24814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:10.842967 24814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:10.843873 24814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:10.845321 24814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:10.845465 24814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:10.845556 24814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:10.845602 24814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:10.878495 24814 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41889
I20260619 06:51:10.878549 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41889 every 8 connection(s)
I20260619 06:51:10.879812 24814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:10.890203 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24814
I20260619 06:51:10.890434 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:10.891091 18589 external_mini_cluster.cc:1428] Setting key 7b8ea38400d81093df9ab8f81b14854a
Jun 19 06:51:10 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851870, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:10.900346 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:10.882393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:59743 (local address 127.18.39.126:40475)
0619 06:51:10.882846 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:10.882851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:10.883906 (+ 1055us) server_negotiation.cc:407] Connection header received
0619 06:51:10.885719 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:10.885724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:10.885792 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:10.885899 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:10.887351 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.888069 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.889670 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.890066 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.890219 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:10.894452 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:10.894458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:10.894463 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:10.894504 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:10.897055 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:10.897917 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:10.897923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:10.897929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:10.898054 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:10.898517 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:10.898529 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:10.898531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:10.898792 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:10.898962 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:10.899728 (+ 766us) server_negotiation.cc:299] Negotiation successful
0619 06:51:10.899969 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":262,"threads_started":1}
I20260619 06:51:10.901438 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:10.882929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:59743)
0619 06:51:10.883702 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:10.883742 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:10.885453 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:10.886105 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:10.886113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:10.886621 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:10.887190 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:10.887207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.888207 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.888210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:10.889535 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:10.889547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:10.890324 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:10.890332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:10.890659 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:10.891456 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:10.891486 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:10.894207 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:10.897229 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:10.897239 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:10.897259 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:10.897781 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:10.898169 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:10.898175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:10.898178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:10.898396 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:10.898936 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:10.898945 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:10.899345 (+ 400us) client_negotiation.cc:769] Sending connection context
0619 06:51:10.899622 (+ 277us) client_negotiation.cc:240] Negotiation successful
0619 06:51:10.900190 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":324,"threads_started":1}
I20260619 06:51:10.903080 24942 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:10.903421 24942 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:10.904181 24942 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:10.906629 24753 ts_manager.cc:194] Registered new tserver with Master: e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:10.909016 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:59743
WARNING: no policy specified for oryx/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
I20260619 06:51:10.940161 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:11.272235 24948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:11.272542 24948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:11.272711 24948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:11.280071 24948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:11.280194 24948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:11.280386 24948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:11.290452 24948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:11.291891 24948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:11.293273 24948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:11.306309 24953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:11.306497 24956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:11.306521 24954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:11.307372 24948 server_base.cc:1034] running on GCE node
I20260619 06:51:11.308033 24948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:11.309026 24948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:11.310256 24948 hybrid_clock.cc:648] HybridClock initialized: now 1781851871310174 us; error 78 us; skew 500 ppm
I20260619 06:51:11.310578 24948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:11 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851871, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:11.317803 24948 init.cc:377] Logged in from keytab as oryx/127.18.39.66@KRBTEST.COM (short username oryx)
I20260619 06:51:11.319509 24948 webserver.cc:460] Webserver started at http://127.18.39.66:38589/ using document root <none> and password file <none>
I20260619 06:51:11.320240 24948 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:11.320390 24948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:11.320722 24948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:11.323714 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "46e1a52c023e41d2b2652bef95dc81ae"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "b8ab49030907c7a7a6b608b0593fee52"
server_key_iv: "d96613775ee15c3633af7a4cfc525555"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.324517 24948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "46e1a52c023e41d2b2652bef95dc81ae"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "b8ab49030907c7a7a6b608b0593fee52"
server_key_iv: "d96613775ee15c3633af7a4cfc525555"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.330546 24948 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 06:51:11.334101 24963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:11.335505 24948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:11.335687 24948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "46e1a52c023e41d2b2652bef95dc81ae"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "b8ab49030907c7a7a6b608b0593fee52"
server_key_iv: "d96613775ee15c3633af7a4cfc525555"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.335897 24948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:11.351441 24948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:11.357861 24948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:11.358278 24948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:11.359709 24948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:11.361378 24948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:11.361472 24948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260619 06:51:11.361568 24948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:11.361615 24948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:11.398486 24948 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:43393
I20260619 06:51:11.398535 25075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:43393 every 8 connection(s)
I20260619 06:51:11.399680 24948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:11.401073 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 24948
I20260619 06:51:11.401211 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:11.401782 18589 external_mini_cluster.cc:1428] Setting key 92816329232ded8d8c9c229a7315c478
Jun 19 06:51:11 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851871, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:51:11.424746 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:11.403661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:38573 (local address 127.18.39.126:40475)
0619 06:51:11.405741 (+ 2080us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:11.405750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:11.405773 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:51:11.406567 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:11.406577 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:11.406671 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:11.406794 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:11.408438 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.409828 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.411747 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.412551 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.412709 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:11.416718 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:11.416729 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:11.416737 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:11.416789 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:11.420862 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:11.421848 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:11.421858 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:11.421867 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:11.422063 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:11.422660 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:11.422668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:11.422672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:11.423115 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:11.423291 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:11.424154 (+ 863us) server_negotiation.cc:299] Negotiation successful
0619 06:51:11.424342 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1937,"thread_start_us":1846,"threads_started":1}
I20260619 06:51:11.425998 25078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:11.403146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:38573)
0619 06:51:11.403944 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:11.403990 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:11.406281 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:11.407046 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:11.407056 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:11.407608 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:11.408230 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:11.408248 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.410017 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.410024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:11.411571 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:11.411584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.412808 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.412814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:11.413040 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:11.413806 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:11.413832 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:11.416402 (+ 2570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:11.421087 (+ 4685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:11.421095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:11.421113 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:11.421697 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:11.422235 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:11.422239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:11.422241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:11.422507 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:11.423282 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:11.423298 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:11.423746 (+ 448us) client_negotiation.cc:769] Sending connection context
0619 06:51:11.424039 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 06:51:11.424491 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":558,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.67@KRBTEST.COM" created.
I20260619 06:51:11.427789 25076 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:11.428196 25076 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:11.428979 25076 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:11.430745 24753 ts_manager.cc:194] Registered new tserver with Master: 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393)
I20260619 06:51:11.432058 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:38573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
I20260619 06:51:11.453320 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:11.785243 25082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:11.785607 25082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:11.785702 25082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:11.792997 25082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:11.793113 25082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:11.793227 25082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:11.803153 25082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:11.804391 25082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:11.805766 25082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:11.819145 25087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:11.819125 25090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:11.819209 25088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:11.820019 25082 server_base.cc:1034] running on GCE node
I20260619 06:51:11.820708 25082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:11.821810 25082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:11.823065 25082 hybrid_clock.cc:648] HybridClock initialized: now 1781851871823023 us; error 52 us; skew 500 ppm
I20260619 06:51:11.823433 25082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:11 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851871, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:11.826999 25082 init.cc:377] Logged in from keytab as oryx/127.18.39.67@KRBTEST.COM (short username oryx)
I20260619 06:51:11.828589 25082 webserver.cc:460] Webserver started at http://127.18.39.67:38971/ using document root <none> and password file <none>
I20260619 06:51:11.829304 25082 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:11.829484 25082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:11.829775 25082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:11.832986 25082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "d0698608120f452dacb282877229b4f6"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "e93db3f1bde71941fa76ad7ee18b54cf"
server_key_iv: "b1ca871a5e494a0fe8dca7e1ad8de22b"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.834010 25082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "d0698608120f452dacb282877229b4f6"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "e93db3f1bde71941fa76ad7ee18b54cf"
server_key_iv: "b1ca871a5e494a0fe8dca7e1ad8de22b"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.839661 25082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:11.843628 25097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:11.845158 25082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 06:51:11.845423 25082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "d0698608120f452dacb282877229b4f6"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "e93db3f1bde71941fa76ad7ee18b54cf"
server_key_iv: "b1ca871a5e494a0fe8dca7e1ad8de22b"
server_key_version: "encryptionkey@0"
I20260619 06:51:11.845713 25082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:11.860517 25082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:11.865563 25082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:11.865886 25082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:11.866925 25082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:11.868269 25082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:11.868342 25082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:11.868495 25082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:11.868553 25082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:11.902454 25082 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:35483
I20260619 06:51:11.902489 25209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:35483 every 8 connection(s)
I20260619 06:51:11.903751 25082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:11.904647 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25082
I20260619 06:51:11.904817 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:11.905421 18589 external_mini_cluster.cc:1428] Setting key c31799db97cd336bd05c8754cba17ee5
I20260619 06:51:11.912523 24942 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
Jun 19 06:51:11 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851871, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:11.924912 25079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:11.906710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:42235 (local address 127.18.39.126:40475)
0619 06:51:11.906880 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:11.906884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:11.908231 (+ 1347us) server_negotiation.cc:407] Connection header received
0619 06:51:11.910226 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:11.910231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:11.910337 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:11.910444 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:11.911952 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.912713 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.914574 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.914976 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.915125 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:11.918697 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:11.918703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:11.918706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:11.918737 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:11.921960 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:11.922754 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:11.922759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:11.922761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:11.922893 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:11.923385 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:11.923389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:11.923391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:11.923688 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:11.923814 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:11.924510 (+ 696us) server_negotiation.cc:299] Negotiation successful
0619 06:51:11.924686 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:51:11.926005 25212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:11.907272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:42235)
0619 06:51:11.908083 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:11.908127 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:11.909962 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:11.910682 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:11.910691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:11.911201 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:11.911787 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:11.911804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.912872 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.912877 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:11.914396 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:11.914408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:11.915208 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:11.915215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:11.915450 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:11.916200 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:11.916224 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:11.918466 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:11.922102 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:11.922110 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:11.922126 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:11.922610 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:11.923010 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:11.923014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:11.923018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:11.923232 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:11.923801 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:11.923807 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:11.924170 (+ 363us) client_negotiation.cc:769] Sending connection context
0619 06:51:11.924397 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 06:51:11.924808 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":370,"threads_started":1}
I20260619 06:51:11.927561 25210 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:11.927862 25210 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:11.928582 25210 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:11.930119 24753 ts_manager.cc:194] Registered new tserver with Master: d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:11.930907 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:42235
I20260619 06:51:11.934439 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.39.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.125.keytab.
Entry for principal oryx/127.18.39.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.125.keytab.
I20260619 06:51:11.982740 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:34959
--webserver_interface=127.18.39.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.125.keytab
--principal=oryx/127.18.39.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40475,127.18.39.125:34959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:12.313166 25215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:12.313494 25215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:12.313586 25215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:12.320916 25215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:12.321040 25215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:12.321089 25215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:12.321144 25215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:12.321187 25215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:12.331471 25215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40475,127.18.39.125:34959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.125.keytab
--principal=oryx/127.18.39.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:34959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.39.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
W20260619 06:51:12.331818 25215 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.39.126:40475,127.18.39.125:34959'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 06:51:12.332886 25215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:12.334333 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:12.346187 25223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:12.346244 25221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:12.346285 25220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:12.347100 25215 server_base.cc:1034] running on GCE node
I20260619 06:51:12.347993 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:12.349498 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:12.350780 25215 hybrid_clock.cc:648] HybridClock initialized: now 1781851872350692 us; error 54 us; skew 500 ppm
I20260619 06:51:12.351135 25215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851872, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:12.355239 25215 init.cc:377] Logged in from keytab as oryx/127.18.39.125@KRBTEST.COM (short username oryx)
I20260619 06:51:12.357309 25215 webserver.cc:460] Webserver started at http://127.18.39.125:37347/ using document root <none> and password file <none>
I20260619 06:51:12.358320 25215 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:12.358464 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:12.358779 25215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:12.361873 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/instance:
uuid: "baf88b00f08b497f9f1decf35a146f1b"
format_stamp: "Formatted at 2026-06-19 06:51:12 on dist-test-slave-1h5s"
server_key: "1efb889f933d9aa627d3b2ac1fde17d7"
server_key_iv: "b8bb3566cfdbc2744ab55d4ef4d48c15"
server_key_version: "encryptionkey@0"
I20260619 06:51:12.362654 25215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal/instance:
uuid: "baf88b00f08b497f9f1decf35a146f1b"
format_stamp: "Formatted at 2026-06-19 06:51:12 on dist-test-slave-1h5s"
server_key: "1efb889f933d9aa627d3b2ac1fde17d7"
server_key_iv: "b8bb3566cfdbc2744ab55d4ef4d48c15"
server_key_version: "encryptionkey@0"
I20260619 06:51:12.367988 25215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:51:12.371546 25230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:12.372932 25215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:12.373122 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "baf88b00f08b497f9f1decf35a146f1b"
format_stamp: "Formatted at 2026-06-19 06:51:12 on dist-test-slave-1h5s"
server_key: "1efb889f933d9aa627d3b2ac1fde17d7"
server_key_iv: "b8bb3566cfdbc2744ab55d4ef4d48c15"
server_key_version: "encryptionkey@0"
I20260619 06:51:12.373370 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:12.399663 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:12.404127 25215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:12.404408 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:12.423084 25215 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.125:34959
I20260619 06:51:12.423090 25281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.125:34959 every 8 connection(s)
I20260619 06:51:12.424472 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
I20260619 06:51:12.428566 25282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:12.434504 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25215
I20260619 06:51:12.435083 25076 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
I20260619 06:51:12.435169 25282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851872, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.125@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
I20260619 06:51:12.457195 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.438284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.18.39.126:40475)
0619 06:51:12.438789 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.438795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.439405 (+ 610us) server_negotiation.cc:407] Connection header received
0619 06:51:12.440963 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.440967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.441036 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.441124 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.442555 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.443334 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.445181 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.445851 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.445944 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.450301 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.450310 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.450315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.450361 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.453581 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.454629 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.454633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.454636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.454765 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.455314 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.455318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.455321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.455636 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.455815 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.456568 (+ 753us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.456812 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":232,"threads_started":1}
I20260619 06:51:12.458403 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.437504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32878 (local address 127.18.39.125:34959)
0619 06:51:12.438375 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.438391 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.438450 (+ 59us) server_negotiation.cc:407] Connection header received
0619 06:51:12.440672 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.440699 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.441017 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.441224 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.442691 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.444287 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.445716 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.446466 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.446595 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.451404 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.451413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.451427 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.451456 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.454715 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.455418 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.455427 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.455432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.455554 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.455960 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.455966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.455969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.456343 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.456662 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.456991 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.457231 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":397,"threads_started":1}
I20260619 06:51:12.458439 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.438332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33724)
0619 06:51:12.439213 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.439267 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.440672 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.441212 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.441220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.442019 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.442380 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.442397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.443484 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.443489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.445039 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.445049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.446081 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.446088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.446320 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.447266 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.447289 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.450065 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.453721 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.453731 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.453745 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.454468 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.454935 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.454940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.454944 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.455148 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.455786 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.455794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.456185 (+ 391us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.456486 (+ 301us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.457037 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":5,"thread_start_us":433,"threads_started":1}
I20260619 06:51:12.460451 25282 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851872, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.125@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
I20260619 06:51:12.472470 25287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.461082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32886)
0619 06:51:12.461320 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.461341 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.461479 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.461855 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.461858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.462167 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.462346 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.462355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.463387 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.463391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.464247 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.464258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.464919 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.464923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.465064 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.465725 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.465742 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.467729 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.470457 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.470461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.470464 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.470833 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.471134 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.471137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.471140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.471280 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.471772 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.471775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.471889 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.472004 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.472224 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 06:51:12.472666 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.461294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.18.39.125:34959)
0619 06:51:12.461468 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.461473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.461491 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:12.461695 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.461699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.461750 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.461821 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.462486 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.463231 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.464377 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.464792 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.464892 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.467917 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.467922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.467925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.467952 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.470316 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.470933 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.470936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.470938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.471046 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.471377 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.471380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.471382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.471643 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.471801 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.472119 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.472344 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":70}
I20260619 06:51:12.477303 25282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Bootstrap starting.
I20260619 06:51:12.480436 25282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:12.481624 25282 log.cc:824] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:12.484488 25282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: No bootstrap required, opened a new log
I20260619 06:51:12.491655 25282 raft_consensus.cc:348] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } }
I20260619 06:51:12.492009 25282 raft_consensus.cc:374] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:12.492132 25282 raft_consensus.cc:729] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf88b00f08b497f9f1decf35a146f1b, State: Initialized, Role: FOLLOWER
I20260619 06:51:12.492781 25282 consensus_queue.cc:260] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } }
I20260619 06:51:12.493697 25289 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } } }
I20260619 06:51:12.494035 25289 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:12.494707 25282 sys_catalog.cc:564] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:51:12.503515 25300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:51:12.503676 25300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:51:12.506155 25215 master_runner.cc:428] Detected that this master baf88b00f08b497f9f1decf35a146f1b is joining an existing cluster
I20260619 06:51:12.506274 25215 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.39.125:34959
I20260619 06:51:12.529518 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.510101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33740 (local address 127.18.39.126:40475)
0619 06:51:12.510318 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.510325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.510965 (+ 640us) server_negotiation.cc:407] Connection header received
0619 06:51:12.511235 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.511242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.511375 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.511470 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.513367 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.514842 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.517539 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.518448 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.518573 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.521913 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.521921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.521925 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.521970 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.525934 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.526906 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.526913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.526918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.527102 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.527587 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.527595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.527601 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.528004 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.528132 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.528766 (+ 634us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.529049 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:51:12.530762 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.510126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33740)
0619 06:51:12.510870 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.510892 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.511070 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.511773 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.511790 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.512624 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.513135 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.513152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.515066 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.515075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.517314 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.517352 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.518963 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.518972 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.519221 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.520036 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.520067 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.521655 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.526087 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.526094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.526100 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.526743 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.527274 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.527280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.527283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.527458 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.528225 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.528233 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.528458 (+ 225us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.529800 (+ 1342us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.530186 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":391,"threads_started":1}
WARNING: no policy specified for oryx/127.18.39.124@KRBTEST.COM; defaulting to no policy
I20260619 06:51:12.546602 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.511031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32896)
0619 06:51:12.525245 (+ 14214us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.525268 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.525648 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.526163 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.526168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.526964 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.527322 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.527338 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.528908 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.528913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.530554 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.530570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.531944 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.531951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.532135 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.532854 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.532876 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.534036 (+ 1160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.538102 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.538108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.538112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.538639 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.544100 (+ 5461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.544109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.544116 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.544393 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.545315 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.545320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.545506 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.545692 (+ 186us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.546074 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14032,"thread_start_us":14352,"threads_started":1}
I20260619 06:51:12.547412 25285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.511449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.18.39.125:34959)
0619 06:51:12.511646 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.511652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.525416 (+ 13764us) server_negotiation.cc:407] Connection header received
0619 06:51:12.525796 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.525802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.525882 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.525985 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.527470 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.528666 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.530706 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.531543 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.531640 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.534220 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.534227 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.534232 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.534265 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.537928 (+ 3663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.538759 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.538770 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.538773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.538920 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.544599 (+ 5679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.544606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.544610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.545080 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.545263 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.546464 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.547063 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.124@KRBTEST.COM" created.
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851870, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.126@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
I20260619 06:51:12.562690 25309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.538737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32904)
0619 06:51:12.545765 (+ 7028us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.545815 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.546044 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.547114 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.547122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.548213 (+ 1091us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.548506 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.548516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.550138 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.550147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.551427 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.551440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.552237 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.552244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.552450 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.553180 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.553212 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.555484 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.559342 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.559359 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.559369 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.559970 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.560437 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.560445 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.560451 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.560625 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.561396 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.561406 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.561627 (+ 221us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.561855 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.562245 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6771,"thread_start_us":6620,"threads_started":1}
I20260619 06:51:12.562752 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.539123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.18.39.125:34959)
0619 06:51:12.545148 (+ 6025us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.545157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.545906 (+ 749us) server_negotiation.cc:407] Connection header received
0619 06:51:12.546330 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.546346 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.546444 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.546544 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.548700 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.549907 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.551594 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.552064 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.552220 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.555744 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.555750 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.555753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.555786 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.559138 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.560119 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.560127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.560134 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.560312 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.560769 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.560776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.560795 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.561218 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.561396 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.561971 (+ 575us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.562216 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5812,"thread_start_us":8407,"threads_started":1}
I20260619 06:51:12.564942 24753 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.39.125:34959: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b"
member_type: NON_VOTER
last_known_addr {
host: "127.18.39.125"
port: 34959
}
attrs {
promote: true
}
}
dest_uuid: "c7bcafeefff94aedad77a5a8a483df88"
cas_config_opid_index: -1
I20260619 06:51:12.566184 24753 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } }
I20260619 06:51:12.576767 25313 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER baf88b00f08b497f9f1decf35a146f1b (127.18.39.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } }
I20260619 06:51:12.578141 25257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.124.keytab.
Entry for principal oryx/127.18.39.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.124.keytab.
I20260619 06:51:12.581954 25313 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.39.125:34959
I20260619 06:51:12.582872 25257 raft_consensus.cc:1261] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Refusing update from remote peer c7bcafeefff94aedad77a5a8a483df88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 06:51:12.583832 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:44217
--webserver_interface=127.18.39.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.124.keytab
--principal=oryx/127.18.39.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:51:12.589838 25215 master.cc:537] Master@127.18.39.125:34959 shutting down...
I20260619 06:51:12.593922 25313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:12.594126 25314 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } }
I20260619 06:51:12.594488 25314 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
W20260619 06:51:12.598205 24739 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 06:51:12.609933 25215 raft_consensus.cc:2229] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:12.610562 25215 raft_consensus.cc:2258] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:12.610801 25215 tablet_replica.cc:330] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: stopping tablet replica
W20260619 06:51:12.616190 24739 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 -> Peer baf88b00f08b497f9f1decf35a146f1b (127.18.39.125:34959): Couldn't send request to peer baf88b00f08b497f9f1decf35a146f1b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 06:51:12.624729 25215 master.cc:552] Master@127.18.39.125:34959 shutdown complete.
I20260619 06:51:12.624845 25215 master_runner.cc:305] Clearing existing system tablet
I20260619 06:51:12.625370 25215 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:12.625484 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:12.629151 25215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 06:51:12.630978 25320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:12.631357 25215 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:12.631630 25215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260619 06:51:12.631711 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "baf88b00f08b497f9f1decf35a146f1b"
format_stamp: "Formatted at 2026-06-19 06:51:12 on dist-test-slave-1h5s"
server_key: "1efb889f933d9aa627d3b2ac1fde17d7"
server_key_iv: "b8bb3566cfdbc2744ab55d4ef4d48c15"
server_key_version: "encryptionkey@0"
I20260619 06:51:12.636852 25215 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:12.644008 25215 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 06:51:12.644209 25215 log.cc:1196] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 06:51:12.644655 25215 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Deleting consensus metadata
I20260619 06:51:12.644928 25215 master_runner.cc:315] Copying system tablet from 127.18.39.126:40475
I20260619 06:51:12.647357 25215 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: tablet copy: Beginning tablet copy session from remote peer at address 127.18.39.126:40475
I20260619 06:51:12.671057 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.648186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33750 (local address 127.18.39.126:40475)
0619 06:51:12.648535 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:12.648541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:12.656366 (+ 7825us) server_negotiation.cc:407] Connection header received
0619 06:51:12.656583 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:12.656590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:12.656679 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:12.656804 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:12.657994 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.659124 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.661032 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.661803 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.661924 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.664259 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:12.664266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:12.664271 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:12.664311 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:12.667883 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.668758 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.668765 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.668769 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.668919 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:12.669427 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:12.669432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:12.669436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:12.669802 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:12.669964 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:12.670443 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 06:51:12.670670 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260619 06:51:12.671069 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:12.648124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33750)
0619 06:51:12.656184 (+ 8060us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:12.656218 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:12.656422 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:12.656862 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:12.656869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:12.657521 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:12.657824 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.657838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.659293 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.659301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.660876 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:12.660891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:12.661992 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:12.661998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:12.662171 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:12.662934 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:12.662956 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:12.664025 (+ 1069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:12.668059 (+ 4034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.668067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.668073 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.668631 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.669072 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:12.669079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:12.669082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:12.669296 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:12.669945 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:12.669949 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:12.670143 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 06:51:12.670310 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 06:51:12.670643 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7895,"thread_start_us":7711,"threads_started":1}
I20260619 06:51:12.672310 24773 tablet_copy_service.cc:140] P c7bcafeefff94aedad77a5a8a483df88: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer baf88b00f08b497f9f1decf35a146f1b ({username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:33750)
I20260619 06:51:12.672582 24773 tablet_copy_service.cc:161] P c7bcafeefff94aedad77a5a8a483df88: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer baf88b00f08b497f9f1decf35a146f1b at {username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:33750: session id = baf88b00f08b497f9f1decf35a146f1b-00000000000000000000000000000000
I20260619 06:51:12.677160 24773 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 06:51:12.679657 25215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 06:51:12.686865 25215 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: tablet copy: Starting download of 0 data blocks...
I20260619 06:51:12.687260 25215 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: tablet copy: Starting download of 1 WAL segments...
I20260619 06:51:12.690340 25215 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 06:51:12.695520 24773 tablet_copy_service.cc:342] P c7bcafeefff94aedad77a5a8a483df88: Request end of tablet copy session baf88b00f08b497f9f1decf35a146f1b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:33750
I20260619 06:51:12.695858 24773 tablet_copy_service.cc:433] P c7bcafeefff94aedad77a5a8a483df88: ending tablet copy session baf88b00f08b497f9f1decf35a146f1b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer baf88b00f08b497f9f1decf35a146f1b
I20260619 06:51:12.708364 25215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:12.710994 25327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:12.714916 25328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:12.722335 25215 server_base.cc:1034] running on GCE node
W20260619 06:51:12.722898 25330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:12.723376 25215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:12.724143 25215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:12.725410 25215 hybrid_clock.cc:648] HybridClock initialized: now 1781851872725350 us; error 40 us; skew 500 ppm
I20260619 06:51:12.725651 25215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:12 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851872, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:12.727906 25215 init.cc:377] Logged in from keytab as oryx/127.18.39.125@KRBTEST.COM (short username oryx)
I20260619 06:51:12.730386 25215 webserver.cc:460] Webserver started at http://127.18.39.125:40337/ using document root <none> and password file <none>
I20260619 06:51:12.730837 25215 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:12.730926 25215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:12.733987 25215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 06:51:12.736018 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:12.736485 25215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260619 06:51:12.736660 25215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "baf88b00f08b497f9f1decf35a146f1b"
format_stamp: "Formatted at 2026-06-19 06:51:12 on dist-test-slave-1h5s"
server_key: "1efb889f933d9aa627d3b2ac1fde17d7"
server_key_iv: "b8bb3566cfdbc2744ab55d4ef4d48c15"
server_key_version: "encryptionkey@0"
I20260619 06:51:12.736861 25215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:12.752576 25215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:12.756682 25215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:12.781785 25215 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.125:34959
I20260619 06:51:12.781996 25388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.125:34959 every 8 connection(s)
I20260619 06:51:12.782827 25215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
I20260619 06:51:12.788255 25389 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:51:12.792271 25389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Bootstrap starting.
I20260619 06:51:12.808465 25389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:51:12.809015 25389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Bootstrap complete.
I20260619 06:51:12.810159 25389 raft_consensus.cc:348] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } }
I20260619 06:51:12.810436 25389 raft_consensus.cc:729] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: baf88b00f08b497f9f1decf35a146f1b, State: Initialized, Role: LEARNER
I20260619 06:51:12.810683 25389 consensus_queue.cc:260] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } }
I20260619 06:51:12.811542 25392 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } }
I20260619 06:51:12.811769 25389 sys_catalog.cc:564] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:12.811959 25392 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: LEARNER
I20260619 06:51:12.826098 25403 catalog_manager.cc:1259] Loaded cluster ID: bc687f4e85d147699224912d02287b09
I20260619 06:51:12.826231 25403 catalog_manager.cc:1552] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: loading cluster ID for follower catalog manager: success
I20260619 06:51:12.829941 25403 catalog_manager.cc:1574] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: acquiring CA information for follower catalog manager: success
I20260619 06:51:12.832451 25403 catalog_manager.cc:1602] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:51:12.933784 25210 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
W20260619 06:51:13.007139 25316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:13.007510 25316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:13.007659 25316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:13.016211 25316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:13.016417 25316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:13.016517 25316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:13.016613 25316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:13.016688 25316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:13.027113 25316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.124.keytab
--principal=oryx/127.18.39.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.39.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:13.028631 25316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:13.030047 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:13.043139 25411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:13.043139 25409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:13.043394 25408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:13.044121 25316 server_base.cc:1034] running on GCE node
I20260619 06:51:13.044942 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:13.046466 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:13.047672 25316 hybrid_clock.cc:648] HybridClock initialized: now 1781851873047629 us; error 37 us; skew 500 ppm
I20260619 06:51:13.048048 25316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:13.051406 25316 init.cc:377] Logged in from keytab as oryx/127.18.39.124@KRBTEST.COM (short username oryx)
I20260619 06:51:13.053310 25316 webserver.cc:460] Webserver started at http://127.18.39.124:45523/ using document root <none> and password file <none>
I20260619 06:51:13.054075 25316 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:13.054219 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:13.054539 25316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:13.057576 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/instance:
uuid: "6de890eb67d249a993a29c8289bbf943"
format_stamp: "Formatted at 2026-06-19 06:51:13 on dist-test-slave-1h5s"
server_key: "ad38587c1f7095cdb786f16ae1322dad"
server_key_iv: "deae989bf09f8d40ab3fa136905f0bd6"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.058341 25316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal/instance:
uuid: "6de890eb67d249a993a29c8289bbf943"
format_stamp: "Formatted at 2026-06-19 06:51:13 on dist-test-slave-1h5s"
server_key: "ad38587c1f7095cdb786f16ae1322dad"
server_key_iv: "deae989bf09f8d40ab3fa136905f0bd6"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.063457 25316 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:13.066815 25418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.068065 25316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 06:51:13.068248 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "6de890eb67d249a993a29c8289bbf943"
format_stamp: "Formatted at 2026-06-19 06:51:13 on dist-test-slave-1h5s"
server_key: "ad38587c1f7095cdb786f16ae1322dad"
server_key_iv: "deae989bf09f8d40ab3fa136905f0bd6"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.068480 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:13.094626 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:13.099203 25316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:13.099520 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:13.119172 25316 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.124:44217
I20260619 06:51:13.119194 25469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.124:44217 every 8 connection(s)
I20260619 06:51:13.120579 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
I20260619 06:51:13.128970 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25316
I20260619 06:51:13.130767 25472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:13.132567 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.120958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32914)
0619 06:51:13.121406 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.121424 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.121533 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.126136 (+ 4603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.126145 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.126210 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:13.126448 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.126460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.128608 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.128615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.130598 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.130612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.131840 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.131847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.132030 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.132086 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.132160 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.132298 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":223,"threads_started":1}
I20260619 06:51:13.133455 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.121142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.18.39.125:34959)
0619 06:51:13.125728 (+ 4586us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.125732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.125749 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:51:13.125811 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.125814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.125910 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.126056 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:13.126842 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.128422 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.130796 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.131591 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.132836 (+ 1245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.133000 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.133042 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.133139 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4466,"thread_start_us":4346,"threads_started":1}
I20260619 06:51:13.134605 25364 raft_consensus.cc:1203] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260619 06:51:13.135146 25392 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcafeefff94aedad77a5a8a483df88. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } }
I20260619 06:51:13.135375 25392 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: LEARNER
I20260619 06:51:13.141956 25472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.124@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:13.156847 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.131323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.18.39.124:44217)
0619 06:51:13.132181 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.132196 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.134410 (+ 2214us) server_negotiation.cc:407] Connection header received
0619 06:51:13.136432 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.136453 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.136930 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.137297 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.139441 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.140790 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.142601 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.143313 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.143472 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.148375 (+ 4903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.148383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.148392 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.148428 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.152342 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.152980 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.152988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.152997 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.153217 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.153631 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.153636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.153640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.154113 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.154425 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.154926 (+ 501us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.155349 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":386,"threads_started":1}
I20260619 06:51:13.159780 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.144968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.18.39.126:40475)
0619 06:51:13.145187 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.145194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.145807 (+ 613us) server_negotiation.cc:407] Connection header received
0619 06:51:13.145999 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.146014 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.146103 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.146214 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.147463 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.148127 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.149750 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.150138 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.150279 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.153836 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.153848 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.153851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.153882 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.157031 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.157811 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.157816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.157820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.157960 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.158342 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.158347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.158350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.158615 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.158762 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.159225 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.159448 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:51:13.159849 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.144859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33756)
0619 06:51:13.145670 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.145716 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.145869 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.146267 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.146272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.146962 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.147280 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.147298 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.148271 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.148276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.149616 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.149628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.150327 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.150333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.150557 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.151092 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.151108 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.153665 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.157188 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.157199 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.157205 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.157687 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.158068 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.158074 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.158078 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.158219 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.158747 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.158759 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.158935 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.159120 (+ 185us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.159435 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":455,"threads_started":1}
I20260619 06:51:13.161046 25472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.124@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
I20260619 06:51:13.174026 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.161654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32916)
0619 06:51:13.161819 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.161835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.161910 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.162358 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.162362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.162697 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.162821 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.162829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.163812 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.163817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.164734 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.164744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.165471 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.165476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.165657 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.166257 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.166273 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.168253 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.171122 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.171126 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.171129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.171548 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.171912 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.171915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.171917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.172044 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.172537 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.172542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.172742 (+ 200us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.172909 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.173686 (+ 777us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 06:51:13.174042 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.161927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.18.39.125:34959)
0619 06:51:13.162111 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.162115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.162129 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:51:13.162175 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.162179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.162234 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.162341 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.162971 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.163673 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.164891 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.165298 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.165462 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.168429 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.168435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.168438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.168471 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.170982 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.171667 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.171671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.171673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.171803 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.172134 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.172138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.172140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.172391 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.172564 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.173013 (+ 449us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.173673 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:51:13.175139 25472 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.124@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
I20260619 06:51:13.187362 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.175760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.0.0.1:47776)
0619 06:51:13.175932 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.175958 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.176047 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.176369 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.176372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.176678 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.176816 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.176823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.177931 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.177935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.178788 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.178799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.179462 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.179466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.179610 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.180184 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.180199 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.182212 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.185170 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.185173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.185176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.185618 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.185945 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.185948 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.185950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.186090 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.186643 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.186646 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.186774 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.186883 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.187065 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 06:51:13.187453 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.175983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.18.39.124:44217)
0619 06:51:13.176143 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.176148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.176163 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:51:13.176204 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.176208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.176260 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.176373 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.176950 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.177735 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.178913 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.179299 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.179399 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.182410 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.182415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.182418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.182446 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.185050 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.185725 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.185729 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.185731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.185850 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.186203 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.186206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.186208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.186543 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.186651 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.186997 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.187173 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:51:13.192866 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Bootstrap starting.
I20260619 06:51:13.196110 25472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:13.197750 25472 log.cc:824] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:13.200407 25472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: No bootstrap required, opened a new log
I20260619 06:51:13.207450 25472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } }
I20260619 06:51:13.207793 25472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:13.207929 25472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de890eb67d249a993a29c8289bbf943, State: Initialized, Role: FOLLOWER
I20260619 06:51:13.208582 25472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } }
I20260619 06:51:13.209756 25479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } } }
I20260619 06:51:13.210141 25479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:13.210882 25472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:51:13.220440 25490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:51:13.220592 25490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:51:13.224381 25316 master_runner.cc:428] Detected that this master 6de890eb67d249a993a29c8289bbf943 is joining an existing cluster
I20260619 06:51:13.224543 25316 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.39.124:44217
I20260619 06:51:13.247779 25473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.229673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.18.39.125:34959)
0619 06:51:13.229877 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.229884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.230571 (+ 687us) server_negotiation.cc:407] Connection header received
0619 06:51:13.231233 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.231239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.231350 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.231441 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.233023 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.234132 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.236220 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.236888 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.236976 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.240319 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.240329 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.240333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.240381 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.244035 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.244894 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.244902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.244905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.245062 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.245582 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.245587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.245593 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.245974 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.246095 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.246711 (+ 616us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.246978 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:51:13.248378 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.231241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.18.39.124:44217)
0619 06:51:13.231478 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.231484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.232088 (+ 604us) server_negotiation.cc:407] Connection header received
0619 06:51:13.232309 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.232314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.232374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.232433 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.233801 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.234917 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.236769 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.237962 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.238072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.240740 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.240746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.240750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.240787 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.244578 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.245387 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.245392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.245395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.245527 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.245914 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.245918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.245922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.246393 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.246527 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.247034 (+ 507us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.247246 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":70}
I20260619 06:51:13.248909 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.229848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32922)
0619 06:51:13.230455 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.230482 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.231098 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.231655 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.231661 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.232428 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.232828 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.232844 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.234321 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.234325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.236058 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.236077 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.237278 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.237295 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.238008 (+ 713us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.238769 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.238799 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.240093 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.244235 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.244244 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.244248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.244753 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.245229 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.245234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.245238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.245434 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.246195 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.246200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.246393 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.248067 (+ 1674us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.248400 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":29,"thread_start_us":277,"threads_started":1}
I20260619 06:51:13.250078 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.230644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.0.0.1:47786)
0619 06:51:13.231973 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.231997 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.232179 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.232642 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.232648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.233333 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.233665 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.233678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.235145 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.235150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.236637 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.236651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.238376 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.238382 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.238583 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.239378 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.239400 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.240583 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.244725 (+ 4142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.244729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.244732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.245248 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.245661 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.245667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.245670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.245795 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.246587 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.246591 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.246746 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.248804 (+ 2058us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.249319 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"threads_started":1}
I20260619 06:51:13.258919 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.228597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.18.39.126:40475)
0619 06:51:13.228766 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.228771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.229499 (+ 728us) server_negotiation.cc:407] Connection header received
0619 06:51:13.229684 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.229690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.229785 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.229877 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.232531 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.233736 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.249669 (+ 15933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.250391 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.250502 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.253167 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.253173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.253176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.253210 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.256081 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.256730 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.256736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.256742 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.256887 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.257293 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.257300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.257303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.257657 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.257823 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.258313 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.258531 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:51:13.258955 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.228668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33766)
0619 06:51:13.229310 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.229398 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.229550 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.230112 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.230118 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.231921 (+ 1803us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.232326 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.232342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.245490 (+ 13148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.245496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.247253 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.247268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.250861 (+ 3593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.250870 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.251052 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.251782 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.251806 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.252902 (+ 1096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.256228 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.256234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.256238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.256594 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.257017 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.257022 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.257026 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.257181 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.257784 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.257789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.258011 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.258166 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.258504 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":350,"threads_started":1}
I20260619 06:51:13.263046 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24814
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851870, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.126@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
I20260619 06:51:13.278939 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.264553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.0.0.1:47794)
0619 06:51:13.264759 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.264778 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.264889 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.265384 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.265391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.265789 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.265934 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.265942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.266953 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.266960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.267836 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.267847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.268605 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.268611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.268789 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.269426 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.269444 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.271504 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.275921 (+ 4417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.275927 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.275931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.276574 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.277023 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.277029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.277033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.277222 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.277924 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.277931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.278105 (+ 174us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.278262 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.278556 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 06:51:13.278973 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.264843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.18.39.124:44217)
0619 06:51:13.265009 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.265013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.265027 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:51:13.265140 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.265145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.265212 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.265329 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.266094 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.266824 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.268021 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.268463 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.268596 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.271697 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.271702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.271705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.271735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.275741 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.276723 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.276729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.276733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.276887 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.277384 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.277390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.277393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.277789 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.277943 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.278366 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.278632 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:51:13.280908 24753 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.39.124:44217: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6de890eb67d249a993a29c8289bbf943"
member_type: NON_VOTER
last_known_addr {
host: "127.18.39.124"
port: 44217
}
attrs {
promote: true
}
}
dest_uuid: "c7bcafeefff94aedad77a5a8a483df88"
cas_config_opid_index: 5
I20260619 06:51:13.282021 24753 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } }
I20260619 06:51:13.284044 25470 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6de890eb67d249a993a29c8289bbf943 (127.18.39.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.287812 25470 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.39.124:44217
I20260619 06:51:13.297283 25316 master.cc:537] Master@127.18.39.124:44217 shutting down...
I20260619 06:51:13.297304 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.297953 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:13.298672 25364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Refusing update from remote peer c7bcafeefff94aedad77a5a8a483df88: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.65@KRBTEST.COM
I20260619 06:51:13.320255 25501 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260619 06:51:13.324098 24739 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 -> Peer 6de890eb67d249a993a29c8289bbf943 (127.18.39.124:44217): Couldn't send request to peer 6de890eb67d249a993a29c8289bbf943. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 06:51:13.324715 25364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6de890eb67d249a993a29c8289bbf943 (127.18.39.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.329258 25392 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: NON_VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: true } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.329639 25392 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: LEARNER
I20260619 06:51:13.330588 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.264449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.18.39.65:41889)
0619 06:51:13.273956 (+ 9507us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.273966 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.273998 (+ 32us) server_negotiation.cc:407] Connection header received
0619 06:51:13.274073 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.274082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.274402 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.276605 (+ 2203us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.276915 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.278324 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.280435 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.281155 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.281376 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.320406 (+ 39030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.320432 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.320454 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.320522 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.326170 (+ 5648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.327233 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.327244 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.327248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.327429 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.327971 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.327977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.327981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.328449 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.328757 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.330051 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.330291 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9284,"thread_start_us":9091,"threads_started":1}
I20260619 06:51:13.331205 25501 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: attempting to promote NON_VOTER baf88b00f08b497f9f1decf35a146f1b to VOTER
I20260619 06:51:13.332171 24937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:13.332070 25501 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } }
I20260619 06:51:13.334738 25364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 LEARNER]: Refusing update from remote peer c7bcafeefff94aedad77a5a8a483df88: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260619 06:51:13.334985 24739 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 -> Peer 6de890eb67d249a993a29c8289bbf943 (127.18.39.124:44217): Couldn't send request to peer 6de890eb67d249a993a29c8289bbf943. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 06:51:13.335364 25500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 06:51:13.338145 25500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, baf88b00f08b497f9f1decf35a146f1b (127.18.39.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.338886 25364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, baf88b00f08b497f9f1decf35a146f1b (127.18.39.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.342101 25470 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.342418 25470 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:13.342134 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.342795 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:13.344448 25316 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:13.344395 25392 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.344647 25392 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:13.344897 25316 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:13.345121 25316 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: stopping tablet replica
I20260619 06:51:13.361686 25316 master.cc:552] Master@127.18.39.124:44217 shutdown complete.
I20260619 06:51:13.361800 25316 master_runner.cc:305] Clearing existing system tablet
I20260619 06:51:13.362143 25316 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:13.362273 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:13.364581 25316 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260619 06:51:13.366080 25507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.366406 25316 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:13.366698 25316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 06:51:13.366794 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "6de890eb67d249a993a29c8289bbf943"
format_stamp: "Formatted at 2026-06-19 06:51:13 on dist-test-slave-1h5s"
server_key: "ad38587c1f7095cdb786f16ae1322dad"
server_key_iv: "deae989bf09f8d40ab3fa136905f0bd6"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.370354 25316 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:13.375389 25316 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 06:51:13.375589 25316 log.cc:1196] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 06:51:13.376075 25316 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Deleting consensus metadata
I20260619 06:51:13.376314 25316 master_runner.cc:315] Copying system tablet from 127.18.39.126:40475
I20260619 06:51:13.379451 25316 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: tablet copy: Beginning tablet copy session from remote peer at address 127.18.39.126:40475
I20260619 06:51:13.397883 25286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.380331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.18.39.126:40475)
0619 06:51:13.380632 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.380642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.385980 (+ 5338us) server_negotiation.cc:407] Connection header received
0619 06:51:13.386212 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.386217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.386284 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.386393 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.387282 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.387984 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.389241 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.389748 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.389900 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.391993 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.391999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.392012 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.392050 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.395188 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.395862 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.395869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.395872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.396010 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:13.396391 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:13.396398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:13.396402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:13.396674 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:13.396844 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.397231 (+ 387us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.397500 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260619 06:51:13.397939 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.380332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:33770)
0619 06:51:13.385855 (+ 5523us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.385872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.386059 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.386435 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.386439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.386905 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.387127 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.387136 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.388154 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.388159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.389076 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.389088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.389910 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.389915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.390121 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.390844 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.390869 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.391743 (+ 874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.395346 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.395351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.395355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.395737 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.396123 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.396132 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.396136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:13.396281 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:13.396809 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:13.396814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:13.396982 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.397143 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.397500 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5383,"thread_start_us":5216,"threads_started":1}
I20260619 06:51:13.398873 24773 tablet_copy_service.cc:140] P c7bcafeefff94aedad77a5a8a483df88: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6de890eb67d249a993a29c8289bbf943 ({username='oryx', principal='oryx/127.18.39.124@KRBTEST.COM'} at 127.0.0.1:33770)
I20260619 06:51:13.399055 24773 tablet_copy_service.cc:161] P c7bcafeefff94aedad77a5a8a483df88: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6de890eb67d249a993a29c8289bbf943 at {username='oryx', principal='oryx/127.18.39.124@KRBTEST.COM'} at 127.0.0.1:33770: session id = 6de890eb67d249a993a29c8289bbf943-00000000000000000000000000000000
I20260619 06:51:13.401297 24773 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 06:51:13.403228 25316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 06:51:13.410826 25316 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: tablet copy: Starting download of 0 data blocks...
I20260619 06:51:13.411144 25316 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: tablet copy: Starting download of 1 WAL segments...
I20260619 06:51:13.413795 25316 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 06:51:13.419193 24773 tablet_copy_service.cc:342] P c7bcafeefff94aedad77a5a8a483df88: Request end of tablet copy session 6de890eb67d249a993a29c8289bbf943-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.39.124@KRBTEST.COM'} at 127.0.0.1:33770
I20260619 06:51:13.419356 24773 tablet_copy_service.cc:433] P c7bcafeefff94aedad77a5a8a483df88: ending tablet copy session 6de890eb67d249a993a29c8289bbf943-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6de890eb67d249a993a29c8289bbf943
I20260619 06:51:13.432102 25316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:13.435003 25514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:13.435438 25515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:13.443392 25316 server_base.cc:1034] running on GCE node
W20260619 06:51:13.443683 25517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:13.444168 25316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:13.444821 25316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:13.446025 25316 hybrid_clock.cc:648] HybridClock initialized: now 1781851873445986 us; error 32 us; skew 500 ppm
I20260619 06:51:13.446718 25316 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:13.449082 25316 init.cc:377] Logged in from keytab as oryx/127.18.39.124@KRBTEST.COM (short username oryx)
I20260619 06:51:13.454665 25316 webserver.cc:460] Webserver started at http://127.18.39.124:40219/ using document root <none> and password file <none>
I20260619 06:51:13.455098 25316 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:13.455214 25316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:13.458600 25316 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 06:51:13.459860 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24814
I20260619 06:51:13.465458 25524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.465916 25316 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.001s
I20260619 06:51:13.466046 25316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "6de890eb67d249a993a29c8289bbf943"
format_stamp: "Formatted at 2026-06-19 06:51:13 on dist-test-slave-1h5s"
server_key: "ad38587c1f7095cdb786f16ae1322dad"
server_key_iv: "deae989bf09f8d40ab3fa136905f0bd6"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.466255 25316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:13.479401 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:41889
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:40475
--webserver_port=41145
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:51:13.490984 25316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:13.495334 25316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:13.524776 25316 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.124:44217
I20260619 06:51:13.524822 25576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.124:44217 every 8 connection(s)
I20260619 06:51:13.525970 25316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
I20260619 06:51:13.531473 25577 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:51:13.535605 25577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Bootstrap starting.
I20260619 06:51:13.555824 25577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:51:13.556361 25577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: Bootstrap complete.
I20260619 06:51:13.557471 25577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } }
I20260619 06:51:13.557766 25577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6de890eb67d249a993a29c8289bbf943, State: Initialized, Role: LEARNER
I20260619 06:51:13.558032 25577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } }
I20260619 06:51:13.558909 25580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.559206 25577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:13.559379 25580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: LEARNER
I20260619 06:51:13.568897 25591 catalog_manager.cc:1259] Loaded cluster ID: bc687f4e85d147699224912d02287b09
I20260619 06:51:13.569018 25591 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: loading cluster ID for follower catalog manager: success
I20260619 06:51:13.572533 25591 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: acquiring CA information for follower catalog manager: success
I20260619 06:51:13.574862 25591 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 06:51:13.845690 25529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:13.846033 25529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:13.846196 25529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:13.854272 25529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:13.854446 25529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:13.854622 25529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:13.865023 25529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:41889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=41145
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:13.866521 25529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:13.867923 25529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:13.881997 25597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:13.882278 25599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:13.883034 25596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:13.883641 25529 server_base.cc:1034] running on GCE node
I20260619 06:51:13.884541 25529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:13.885766 25529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:13.887007 25529 hybrid_clock.cc:648] HybridClock initialized: now 1781851873886937 us; error 64 us; skew 500 ppm
I20260619 06:51:13.887338 25529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:13.889623 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.882110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.0.0.1:47806)
0619 06:51:13.882736 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.882754 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.882989 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.883825 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.883831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.883858 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:13.884024 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.884032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.885820 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.885825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.887348 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.887362 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.888750 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.888756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.888918 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.888951 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:51:13.889078 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 06:51:13.889238 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":388,"threads_started":1}
I20260619 06:51:13.889622 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.882730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.18.39.124:44217)
0619 06:51:13.883398 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.883403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.883419 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:51:13.883511 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.883516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.883613 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.883743 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:13.884161 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.885668 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.887502 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.888566 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.888728 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.888955 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:13.889101 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 06:51:13.889238 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":392,"threads_started":1}
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:13.890643 25552 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 06:51:13.891476 25529 init.cc:377] Logged in from keytab as oryx/127.18.39.65@KRBTEST.COM (short username oryx)
I20260619 06:51:13.891409 25580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bcafeefff94aedad77a5a8a483df88. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:13.891727 25580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: LEARNER
I20260619 06:51:13.893193 25529 webserver.cc:460] Webserver started at http://127.18.39.65:41145/ using document root <none> and password file <none>
I20260619 06:51:13.894002 25529 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:13.894150 25529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:13.899613 25529 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 06:51:13.902644 25608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.904188 25529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:51:13.904354 25529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "e467617eed594e9a9611d0a3b1bdd644"
format_stamp: "Formatted at 2026-06-19 06:51:10 on dist-test-slave-1h5s"
server_key: "51a489ae2af23ab9f5b092d2313eaf60"
server_key_iv: "0c54060586f21780510ba49065c4e1b4"
server_key_version: "encryptionkey@0"
I20260619 06:51:13.905335 25529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:13.927639 25529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:13.932358 25529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:13.932657 25529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:13.933678 25529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:13.935086 25529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:13.935200 25529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.935333 25529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:13.935427 25529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:13.972792 25529 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41889
I20260619 06:51:13.972878 25720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41889 every 8 connection(s)
I20260619 06:51:13.974536 25529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:13.982836 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25529
I20260619 06:51:13.983209 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24948
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
Jun 19 06:51:13 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851873, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:14.004199 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.980346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:57685 (local address 127.18.39.124:44217)
0619 06:51:13.980531 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.980539 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.980817 (+ 278us) server_negotiation.cc:407] Connection header received
0619 06:51:13.983044 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.983053 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.983141 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.983205 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.985292 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.986877 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.989437 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.989860 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.990009 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.995660 (+ 5651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.995668 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.995675 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.995721 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.999872 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.000997 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.001002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.001006 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.001390 (+ 384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.001996 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.002001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.002004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.002337 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.002470 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.003593 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.003825 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 06:51:14.004221 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.978152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42395 (local address 127.18.39.126:40475)
0619 06:51:13.978598 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.978605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.979692 (+ 1087us) server_negotiation.cc:407] Connection header received
0619 06:51:13.982599 (+ 2907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.982605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.982691 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.982802 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.986760 (+ 3958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.987909 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.991897 (+ 3988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.992598 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.992744 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.995858 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.995863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.995866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.995898 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:13.999764 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.001004 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.001009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.001013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.001163 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.002003 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.002008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.002011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.002337 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.002481 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.003593 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.003825 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":200,"threads_started":1}
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.66@KRBTEST.COM
I20260619 06:51:14.006151 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.980116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.18.39.65:57685)
0619 06:51:13.980693 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.980721 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.982213 (+ 1492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.983425 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.983428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.984077 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.985052 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.985084 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.987072 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.987078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.989263 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.989276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.990121 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.990128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.990554 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.991408 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.991446 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.995382 (+ 3936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:14.000012 (+ 4630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.000016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.000019 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.000852 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.001568 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.001572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.001574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.001858 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.002567 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.002575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.003149 (+ 574us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.004501 (+ 1352us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.004770 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":135,"spinlock_wait_cycles":18560,"thread_start_us":432,"threads_started":1}
I20260619 06:51:14.006175 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.978827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:42395)
0619 06:51:13.979526 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.979567 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.982182 (+ 2615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.983099 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.983112 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.983949 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.986607 (+ 2658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.986616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.988086 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.988090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.991757 (+ 3667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.991768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.992815 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.992825 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.993069 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.993710 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.993739 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.995641 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:13.999949 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:13.999960 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:13.999982 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.000852 (+ 870us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.001549 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.001555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.001559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.001858 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.002567 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.002575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.003149 (+ 574us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.004970 (+ 1821us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.005272 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":2201,"spinlock_wait_cycles":22144,"thread_start_us":279,"threads_started":1}
I20260619 06:51:14.006726 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.981029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.18.39.65:44847)
0619 06:51:13.981654 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:13.981680 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:13.982203 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:13.985048 (+ 2845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:13.985051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:13.985614 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:13.986334 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.986350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.988284 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.988293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.990614 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:13.990627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.991511 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.991517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:13.991720 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.992360 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:13.992376 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:13.997415 (+ 5039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:14.001379 (+ 3964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.001384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.001387 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.001942 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.002389 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.002397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.002401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.002584 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.003359 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.003366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.003537 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.003713 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.004289 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":7,"thread_start_us":372,"threads_started":1}
I20260619 06:51:14.007406 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.981042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:44847 (local address 127.18.39.125:34959)
0619 06:51:13.981889 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.981893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.981912 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:13.982609 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.982614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.982696 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.982802 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.986525 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.988061 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.990788 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.991335 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.991457 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:13.997681 (+ 6224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:13.997692 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:13.997698 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:13.997741 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.001185 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.002090 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.002095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.002099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.002238 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.002729 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.002736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.002740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.003111 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.003260 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.006201 (+ 2941us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.006997 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":602,"threads_started":1}
I20260619 06:51:14.007993 25722 heartbeater.cc:347] Connected to a master server at 127.18.39.124:44217
I20260619 06:51:14.008086 25721 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:14.008350 25722 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.008759 25723 heartbeater.cc:347] Connected to a master server at 127.18.39.125:34959
I20260619 06:51:14.008932 25723 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.008348 25721 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.009966 25723 heartbeater.cc:511] Master 127.18.39.125:34959 requested a full tablet report, sending...
I20260619 06:51:14.009927 25722 heartbeater.cc:511] Master 127.18.39.124:44217 requested a full tablet report, sending...
I20260619 06:51:14.009990 25721 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:14.012044 24753 ts_manager.cc:194] Re-registered known tserver with Master: e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:14.012404 25354 ts_manager.cc:194] Registered new tserver with Master: e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:14.012730 25542 ts_manager.cc:194] Registered new tserver with Master: e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:14.013154 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:42395
I20260619 06:51:14.014533 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:13.984217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.18.39.66:43393)
0619 06:51:13.991284 (+ 7067us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:13.991336 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:13.991371 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:51:13.991441 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:13.991450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:13.991793 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:13.995283 (+ 3490us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:13.995564 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:13.997551 (+ 1987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:13.999613 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.000588 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.001318 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.007126 (+ 5808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:14.007142 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:14.007160 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:14.007205 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.011185 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.012292 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.012302 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.012318 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.012522 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.012983 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.012987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.012990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.013431 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.013759 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.014061 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.014257 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6508,"thread_start_us":6294,"threads_started":1}
I20260619 06:51:14.015710 25071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:14.100370 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24948
I20260619 06:51:14.117688 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:43393
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:40475
--webserver_port=38589
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:51:14.445958 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: NON_VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: true } } }
I20260619 06:51:14.446357 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:14.447777 25734 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: attempting to promote NON_VOTER 6de890eb67d249a993a29c8289bbf943 to VOTER
I20260619 06:51:14.448323 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } }
I20260619 06:51:14.450641 25364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Refusing update from remote peer c7bcafeefff94aedad77a5a8a483df88: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 06:51:14.450706 25552 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 LEARNER]: Refusing update from remote peer c7bcafeefff94aedad77a5a8a483df88: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 06:51:14.451287 25734 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 06:51:14.451699 25735 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 06:51:14.454950 25500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6de890eb67d249a993a29c8289bbf943 (127.18.39.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.455641 25364 raft_consensus.cc:2932] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6de890eb67d249a993a29c8289bbf943 (127.18.39.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.458827 25735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.459095 25735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:14.459394 25735 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.459698 25735 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
W20260619 06:51:14.463408 25733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:14.463724 25733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:14.463884 25733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 06:51:14.471659 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.471982 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:14.471791 25743 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.472162 25743 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: FOLLOWER
W20260619 06:51:14.472731 25733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:14.472833 25733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:14.472994 25733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:14.472723 25552 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 6de890eb67d249a993a29c8289bbf943 (127.18.39.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.476332 25744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7bcafeefff94aedad77a5a8a483df88" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:14.476758 25744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:14.483568 25733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:43393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=38589
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:14.484885 25733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:14.486217 25733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:14.500842 25746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:14.500934 25749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:14.500952 25747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:14.501662 25733 server_base.cc:1034] running on GCE node
I20260619 06:51:14.502245 25733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:14.503188 25733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:14.504384 25733 hybrid_clock.cc:648] HybridClock initialized: now 1781851874504322 us; error 53 us; skew 500 ppm
I20260619 06:51:14.504742 25733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851874, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:14.508496 25733 init.cc:377] Logged in from keytab as oryx/127.18.39.66@KRBTEST.COM (short username oryx)
I20260619 06:51:14.510056 25733 webserver.cc:460] Webserver started at http://127.18.39.66:38589/ using document root <none> and password file <none>
I20260619 06:51:14.510849 25733 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:14.510963 25733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:14.516624 25733 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 06:51:14.519834 25756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:14.521600 25733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:51:14.521778 25733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "46e1a52c023e41d2b2652bef95dc81ae"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "b8ab49030907c7a7a6b608b0593fee52"
server_key_iv: "d96613775ee15c3633af7a4cfc525555"
server_key_version: "encryptionkey@0"
I20260619 06:51:14.522787 25733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:14.562645 25733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:14.566823 25733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:14.567056 25733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:14.568077 25733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:14.569480 25733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:14.569567 25733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:14.569646 25733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:14.569701 25733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:14.603289 25733 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:43393
I20260619 06:51:14.603384 25868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:43393 every 8 connection(s)
I20260619 06:51:14.604890 25733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:14.608754 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25733
I20260619 06:51:14.609117 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25082
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851874, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
I20260619 06:51:14.637838 25873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.607971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:57315 (local address 127.18.39.125:34959)
0619 06:51:14.608422 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:14.608427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:14.609694 (+ 1267us) server_negotiation.cc:407] Connection header received
0619 06:51:14.613040 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:14.613050 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:14.613180 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:14.613295 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:14.616142 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.617741 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.621626 (+ 3885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.622518 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.622646 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.627574 (+ 4928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:14.627584 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:14.627591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:14.627652 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.632597 (+ 4945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.634575 (+ 1978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.634584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.634590 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.634756 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.635308 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.635313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.635316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.635719 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.635862 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.637148 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.637437 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":238,"threads_started":1}
I20260619 06:51:14.640138 25874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.608534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.18.39.66:57315)
0619 06:51:14.609489 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:14.609558 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:14.612630 (+ 3072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:14.613995 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:14.614011 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:14.614942 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:14.615895 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.615925 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.618033 (+ 2108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.618041 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.621402 (+ 3361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.621424 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.623094 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.623104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.623539 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.624496 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:14.624525 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:14.627288 (+ 2763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:14.632840 (+ 5552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.632855 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.632881 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.634413 (+ 1532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.634890 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.634895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.634898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.635160 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.635950 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.635959 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.636416 (+ 457us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.636792 (+ 376us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.638495 (+ 1703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":1054,"thread_start_us":351,"threads_started":1}
I20260619 06:51:14.642199 25871 heartbeater.cc:347] Connected to a master server at 127.18.39.125:34959
I20260619 06:51:14.642556 25871 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.643553 25871 heartbeater.cc:511] Master 127.18.39.125:34959 requested a full tablet report, sending...
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.67@KRBTEST.COM
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851874, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
I20260619 06:51:14.645606 25354 ts_manager.cc:194] Registered new tserver with Master: 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393)
Jun 19 06:51:14 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851874, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:14.653683 25878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.610568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:51291 (local address 127.18.39.124:44217)
0619 06:51:14.627976 (+ 17408us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:14.627983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:14.628566 (+ 583us) server_negotiation.cc:407] Connection header received
0619 06:51:14.632658 (+ 4092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:14.632666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:14.632761 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:14.634802 (+ 2041us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:14.635055 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.636253 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.639113 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.639865 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.639995 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.646603 (+ 6608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:14.646610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:14.646615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:14.646663 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.650363 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.651276 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.651283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.651288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.651439 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.651844 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.651849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.651853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.652240 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.652382 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.653063 (+ 681us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.653265 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17248,"thread_start_us":17091,"threads_started":1}
I20260619 06:51:14.653714 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.610057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.18.39.66:51291)
0619 06:51:14.628454 (+ 18397us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:14.628475 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:14.628618 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:14.633207 (+ 4589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:14.633212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:14.633882 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:14.634228 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.634239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.636411 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.636416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.638965 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.638980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.640570 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.640581 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.641697 (+ 1116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.642732 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:14.642759 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:14.646340 (+ 3581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:14.650490 (+ 4150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.650495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.650499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.651114 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.651560 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.651565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.651568 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.651708 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.652662 (+ 954us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.652666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.652797 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.652951 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.653254 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18251,"mutex_wait_us":31,"spinlock_wait_cycles":896,"thread_start_us":18078,"threads_started":1}
I20260619 06:51:14.654961 25870 heartbeater.cc:347] Connected to a master server at 127.18.39.124:44217
I20260619 06:51:14.655068 25880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.628883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:44043 (local address 127.18.39.126:40475)
0619 06:51:14.638436 (+ 9553us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:14.638442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:14.638462 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:51:14.638548 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:14.638553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:14.638626 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:14.638731 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:14.640300 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.641324 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.643866 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.644515 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.644669 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.648442 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:14.648448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:14.648452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:14.648503 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.651698 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.652383 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.652388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.652393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.652557 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.653443 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.653452 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.653458 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.653794 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.653949 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.654446 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.654680 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9419,"thread_start_us":9313,"threads_started":1}
I20260619 06:51:14.655140 25870 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.655138 25879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.628747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:44043)
0619 06:51:14.634526 (+ 5779us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:14.634549 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:14.634648 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:14.639156 (+ 4508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:14.639161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:14.639773 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:14.640070 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.640089 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.641519 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.641524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.643671 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:14.643687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.644717 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.644723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:14.644902 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.645539 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:14.645562 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:14.648254 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:14.651827 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.651834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.651840 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.652252 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.652684 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:14.652689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:14.652693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:14.652849 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:14.654012 (+ 1163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:14.654017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:14.654162 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 06:51:14.654303 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 06:51:14.654730 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5652,"thread_start_us":5487,"threads_started":1}
I20260619 06:51:14.655570 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:14.610157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52590 (local address 127.18.39.67:35483)
0619 06:51:14.629184 (+ 19027us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:14.629197 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:14.629235 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:51:14.629315 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:14.629327 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:14.629702 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:14.629897 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:14.632810 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.634573 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.637094 (+ 2521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:14.638614 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:14.638782 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:14.646217 (+ 7435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:14.646236 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:14.646252 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:14.646302 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:14.650782 (+ 4480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.651626 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.651635 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.651639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.651804 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:14.652412 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:14.652417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:14.652420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:14.652831 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:14.653098 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:14.654814 (+ 1716us) server_negotiation.cc:299] Negotiation successful
0619 06:51:14.655171 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18803,"thread_start_us":23624,"threads_started":1}
I20260619 06:51:14.655647 25870 heartbeater.cc:511] Master 127.18.39.124:44217 requested a full tablet report, sending...
I20260619 06:51:14.656749 25542 ts_manager.cc:194] Registered new tserver with Master: 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393)
I20260619 06:51:14.656865 25205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:14.657135 25869 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:14.657263 25869 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:14.657677 25869 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:14.658718 24753 ts_manager.cc:194] Re-registered known tserver with Master: 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393)
I20260619 06:51:14.659618 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:44043
I20260619 06:51:14.747258 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25082
I20260619 06:51:14.763827 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:35483
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:40475
--webserver_port=38971
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:51:15.015617 25721 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
W20260619 06:51:15.111092 25882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:15.111413 25882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:15.111557 25882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:15.119253 25882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:15.119378 25882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:15.119493 25882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:15.129701 25882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:35483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=38971
--tserver_master_addrs=127.18.39.126:40475,127.18.39.125:34959,127.18.39.124:44217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:15.130996 25882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:15.132491 25882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:15.146325 25887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:15.146446 25888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:15.146462 25890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:15.147467 25882 server_base.cc:1034] running on GCE node
I20260619 06:51:15.148145 25882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:15.149231 25882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:15.150527 25882 hybrid_clock.cc:648] HybridClock initialized: now 1781851875150482 us; error 50 us; skew 500 ppm
I20260619 06:51:15.150882 25882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:15 dist-test-slave-1h5s krb5kdc[24708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851875, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:15.154358 25882 init.cc:377] Logged in from keytab as oryx/127.18.39.67@KRBTEST.COM (short username oryx)
I20260619 06:51:15.156026 25882 webserver.cc:460] Webserver started at http://127.18.39.67:38971/ using document root <none> and password file <none>
I20260619 06:51:15.156738 25882 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:15.156859 25882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:15.163487 25882 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:51:15.166608 25897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:15.168236 25882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 06:51:15.168444 25882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "d0698608120f452dacb282877229b4f6"
format_stamp: "Formatted at 2026-06-19 06:51:11 on dist-test-slave-1h5s"
server_key: "e93db3f1bde71941fa76ad7ee18b54cf"
server_key_iv: "b1ca871a5e494a0fe8dca7e1ad8de22b"
server_key_version: "encryptionkey@0"
I20260619 06:51:15.169624 25882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:15.197391 25882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:15.201393 25882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:15.201694 25882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:15.202690 25882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:15.204027 25882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:15.204138 25882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:15.204264 25882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:15.204362 25882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:15.238667 25882 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:35483
I20260619 06:51:15.238719 26009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:35483 every 8 connection(s)
I20260619 06:51:15.240083 25882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:15.245620 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 25882
Jun 19 06:51:15 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851875, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
Jun 19 06:51:15 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851875, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.125@KRBTEST.COM
Jun 19 06:51:15 dist-test-slave-1h5s krb5kdc[24708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851875, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.124@KRBTEST.COM
I20260619 06:51:15.271801 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.245020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:50159 (local address 127.18.39.126:40475)
0619 06:51:15.245762 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.245769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.246546 (+ 777us) server_negotiation.cc:407] Connection header received
0619 06:51:15.247829 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.247838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.247963 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.248109 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.250700 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.251804 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.254924 (+ 3120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.255614 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.255748 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.260732 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.260739 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.260747 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.260793 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.265247 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.266690 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.266696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.266700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.266913 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.269476 (+ 2563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.269484 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.269488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.269889 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.270040 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.271112 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.271388 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":516,"threads_started":1}
I20260619 06:51:15.273598 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.244915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:50159)
0619 06:51:15.246140 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.246158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.247540 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.248356 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.248369 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.249903 (+ 1534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:15.250473 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.250499 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.252248 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.252258 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.254781 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.254796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.255830 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.255838 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.256145 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.256944 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:15.256984 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:15.260509 (+ 3525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:15.265708 (+ 5199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.265727 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.265759 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.266567 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.268957 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.268966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.268973 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.269311 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.270030 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:15.270043 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:15.270622 (+ 579us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.271001 (+ 379us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.271754 (+ 753us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":236,"thread_start_us":923,"threads_started":1}
I20260619 06:51:15.274212 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.246722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:47493 (local address 127.18.39.125:34959)
0619 06:51:15.247373 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.247380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.247398 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:15.248035 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.248041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.248805 (+ 764us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.249192 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.251025 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.252044 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.255993 (+ 3949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.257201 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.257294 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.266232 (+ 8938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.266240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.266245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.266299 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.270037 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.271595 (+ 1558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.271603 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.271607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.271783 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.272283 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.272290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.272295 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.272665 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.272816 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.273477 (+ 661us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.273800 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":436,"threads_started":1}
I20260619 06:51:15.275339 26010 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40475
I20260619 06:51:15.275454 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.246374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.18.39.67:47493)
0619 06:51:15.246940 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.246957 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.247884 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.249025 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.249032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.249690 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:15.250495 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.250520 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.252189 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.252197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.254781 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.254796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.257659 (+ 2863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.257665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.257874 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.258562 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:15.258591 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:15.265966 (+ 7375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:15.270848 (+ 4882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.270856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.270862 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.271412 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.271949 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.271956 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.271961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.272148 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.272899 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:15.272904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:15.273089 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.274628 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.274988 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":348,"thread_start_us":326,"threads_started":1}
I20260619 06:51:15.275682 26010 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:15.276345 26012 heartbeater.cc:347] Connected to a master server at 127.18.39.125:34959
I20260619 06:51:15.276499 26012 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:15.277180 26010 heartbeater.cc:511] Master 127.18.39.126:40475 requested a full tablet report, sending...
I20260619 06:51:15.277487 26012 heartbeater.cc:511] Master 127.18.39.125:34959 requested a full tablet report, sending...
I20260619 06:51:15.278733 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.244081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:39021 (local address 127.18.39.124:44217)
0619 06:51:15.245385 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.245392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.245418 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:51:15.247864 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.247870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.247974 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.248054 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.250674 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.251787 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.255002 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.255711 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.255805 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.263186 (+ 7381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.263196 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.263203 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.263247 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.268765 (+ 5518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.275403 (+ 6638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.275409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.275416 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.275607 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.276119 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.276125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.276128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.276520 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.276661 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.277697 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.277946 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":1054,"threads_started":1}
I20260619 06:51:15.278998 24753 ts_manager.cc:194] Re-registered known tserver with Master: d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:15.279281 25354 ts_manager.cc:194] Registered new tserver with Master: d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:15.279637 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.243526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.18.39.67:39021)
0619 06:51:15.244472 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.244539 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.247540 (+ 3001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.248356 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.248369 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.249313 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:15.250501 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.250510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.252015 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.252024 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.254817 (+ 2793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.254835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.256094 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.256100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.256301 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.258884 (+ 2583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:15.258912 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:15.262904 (+ 3992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:15.269498 (+ 6594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.269505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.269511 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.274617 (+ 5106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.275778 (+ 1161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:15.275783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:15.275786 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:15.275975 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:15.277142 (+ 1167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:15.277148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:15.277383 (+ 235us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.278912 (+ 1529us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.279225 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":668,"thread_start_us":343,"threads_started":1}
I20260619 06:51:15.280143 24753 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:50159
I20260619 06:51:15.281385 26011 heartbeater.cc:347] Connected to a master server at 127.18.39.124:44217
I20260619 06:51:15.281540 26011 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:15.282286 26011 heartbeater.cc:511] Master 127.18.39.124:44217 requested a full tablet report, sending...
I20260619 06:51:15.283394 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.251556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33778 (local address 127.18.39.126:40475)
0619 06:51:15.255209 (+ 3653us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.255215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.256781 (+ 1566us) server_negotiation.cc:407] Connection header received
0619 06:51:15.256838 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.256844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.256922 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.257036 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.259679 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.260689 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.263168 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.264377 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.264475 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.270232 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.270236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.270239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.270267 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.276010 (+ 5743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.277139 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.277145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.277151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.277405 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.277948 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.277954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.277959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.278324 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.278604 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.280181 (+ 1577us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.280418 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3511,"thread_start_us":3405,"threads_started":1}
I20260619 06:51:15.283767 25542 ts_manager.cc:194] Registered new tserver with Master: d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:15.299011 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.256255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32924 (local address 127.18.39.125:34959)
0619 06:51:15.274584 (+ 18329us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.274591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.279986 (+ 5395us) server_negotiation.cc:407] Connection header received
0619 06:51:15.280184 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.280190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.280279 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.280347 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.282904 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.284136 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.286274 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.287049 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.287199 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.291511 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.291518 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.291523 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.291570 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.295489 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.296251 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.296263 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.296266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.296393 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.296806 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.296811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.296813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.297091 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.297501 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.298151 (+ 650us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.298334 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18224,"thread_start_us":28043,"threads_started":1}
I20260619 06:51:15.290623 24753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:15.301095 24753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:15.301507 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.285150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.18.39.124:44217)
0619 06:51:15.285358 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.285366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.285700 (+ 334us) server_negotiation.cc:407] Connection header received
0619 06:51:15.285842 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.285846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.285922 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.286022 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.288234 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.289062 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.291223 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.291889 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.292035 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.294559 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.294565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.294570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.294612 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.297947 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.299078 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.299083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.299086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.299249 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.299680 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.299685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.299689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.300084 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.300352 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.300895 (+ 543us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.301126 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:51:15.349154 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.320333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.18.39.66:43393)
0619 06:51:15.337723 (+ 17390us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.337736 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.337776 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:51:15.337905 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.337918 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.338293 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.338542 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:15.340097 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.342147 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.345717 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.347176 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.347295 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.347515 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.348656 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.348840 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17134,"thread_start_us":16980,"threads_started":1}
I20260619 06:51:15.349203 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.320552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43393 (local address 127.0.0.1:42958)
0619 06:51:15.334965 (+ 14413us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.334991 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.335156 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.338849 (+ 3693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.338858 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.338896 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:15.339477 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.339500 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.342295 (+ 2795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.342300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.345538 (+ 3238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.345553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.348229 (+ 2676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.348234 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.348437 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.348480 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.348587 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.348747 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14229,"mutex_wait_us":1,"thread_start_us":20971,"threads_started":1}
I20260619 06:51:15.351052 26036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.321707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41889 (local address 127.0.0.1:40836)
0619 06:51:15.341735 (+ 20028us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.341753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.341903 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.342826 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.342831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.342857 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:15.343342 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.343354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.345996 (+ 2642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.346002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.348178 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.348192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.350310 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.350316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.350478 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.350508 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.350613 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.350747 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19927,"thread_start_us":12877,"threads_started":1}
I20260619 06:51:15.351151 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.321972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.18.39.65:41889)
0619 06:51:15.333641 (+ 11669us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.333654 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.341874 (+ 8220us) server_negotiation.cc:407] Connection header received
0619 06:51:15.342081 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.342092 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.342348 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.342550 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:15.343571 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.345809 (+ 2238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.348321 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.349826 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.349957 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.350163 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.350672 (+ 509us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.350852 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11195,"thread_start_us":11351,"threads_started":1}
I20260619 06:51:15.354876 26041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.341921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:35483 (local address 127.0.0.1:52602)
0619 06:51:15.343386 (+ 1465us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.343409 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.343595 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.344451 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.344456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.344479 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:15.346180 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.346195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.348371 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.348376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.350623 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.350637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.353647 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.353656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.353866 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.353905 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.354005 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.354144 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"thread_start_us":1175,"threads_started":1}
I20260619 06:51:15.355468 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.342136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.18.39.67:35483)
0619 06:51:15.342784 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.342796 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.343493 (+ 697us) server_negotiation.cc:407] Connection header received
0619 06:51:15.343757 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.343764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.343992 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.344149 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:15.346412 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.348063 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.350803 (+ 2740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.351936 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.352057 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.352255 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.354090 (+ 1835us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.355156 (+ 1066us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":313,"threads_started":1}
I20260619 06:51:15.355055 25656 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc6ea95a335439f889b5213f65bc67d (DEFAULT_TABLE table=test-table [id=9e657be818734d9b8f975befd83855fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:15.355055 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc6ea95a335439f889b5213f65bc67d (DEFAULT_TABLE table=test-table [id=9e657be818734d9b8f975befd83855fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:15.356510 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc6ea95a335439f889b5213f65bc67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:15.356510 25656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc6ea95a335439f889b5213f65bc67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:15.358798 25945 tablet_service.cc:1467] Processing CreateTablet for tablet 1fc6ea95a335439f889b5213f65bc67d (DEFAULT_TABLE table=test-table [id=9e657be818734d9b8f975befd83855fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:15.360109 25945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fc6ea95a335439f889b5213f65bc67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:15.371901 26042 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Bootstrap starting.
I20260619 06:51:15.375342 26042 tablet_bootstrap.cc:654] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:15.376332 26042 log.cc:824] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:15.378578 26043 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Bootstrap starting.
I20260619 06:51:15.378595 26044 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Bootstrap starting.
I20260619 06:51:15.382630 26043 tablet_bootstrap.cc:654] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:15.383818 26044 tablet_bootstrap.cc:654] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:15.383767 26043 log.cc:824] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:15.384992 26044 log.cc:824] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:15.385236 26042 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: No bootstrap required, opened a new log
I20260619 06:51:15.385560 26042 ts_tablet_manager.cc:1397] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260619 06:51:15.387035 26043 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: No bootstrap required, opened a new log
I20260619 06:51:15.387293 26043 ts_tablet_manager.cc:1397] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 06:51:15.388180 26044 tablet_bootstrap.cc:492] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: No bootstrap required, opened a new log
I20260619 06:51:15.388427 26044 ts_tablet_manager.cc:1397] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 06:51:15.394834 26043 raft_consensus.cc:348] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.395223 26043 raft_consensus.cc:374] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:15.395316 26043 raft_consensus.cc:729] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e467617eed594e9a9611d0a3b1bdd644, State: Initialized, Role: FOLLOWER
I20260619 06:51:15.395955 26043 consensus_queue.cc:260] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.396445 26042 raft_consensus.cc:348] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.396844 26042 raft_consensus.cc:374] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:15.396940 26042 raft_consensus.cc:729] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0698608120f452dacb282877229b4f6, State: Initialized, Role: FOLLOWER
I20260619 06:51:15.397742 26043 ts_tablet_manager.cc:1428] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Time spent starting tablet: real 0.010s user 0.006s sys 0.006s
I20260619 06:51:15.397675 26042 consensus_queue.cc:260] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.399696 26044 raft_consensus.cc:348] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.400238 26044 raft_consensus.cc:374] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:15.400341 26044 raft_consensus.cc:729] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e1a52c023e41d2b2652bef95dc81ae, State: Initialized, Role: FOLLOWER
I20260619 06:51:15.400990 26044 consensus_queue.cc:260] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.401242 26010 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
I20260619 06:51:15.402083 26042 ts_tablet_manager.cc:1428] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260619 06:51:15.403546 25869 heartbeater.cc:503] Master 127.18.39.126:40475 was elected leader, sending a full tablet report...
I20260619 06:51:15.404220 26044 ts_tablet_manager.cc:1428] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
W20260619 06:51:15.479563 25724 tablet.cc:2369] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:15.494161 26013 tablet.cc:2369] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:15.608645 25872 tablet.cc:2369] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:51:15.747152 26050 raft_consensus.cc:482] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:15.747481 26050 raft_consensus.cc:504] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.749174 26050 leader_election.cc:288] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0698608120f452dacb282877229b4f6 (127.18.39.67:35483), e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:15.756532 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.749634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:33093 (local address 127.18.39.65:41889)
0619 06:51:15.749800 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.749804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.750555 (+ 751us) server_negotiation.cc:407] Connection header received
0619 06:51:15.750804 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.750812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.750892 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.750959 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:15.751595 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.752300 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.754186 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.755084 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.755210 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.755298 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.756121 (+ 823us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.756250 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 06:51:15.756525 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.749506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:37449 (local address 127.18.39.67:35483)
0619 06:51:15.749714 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.749723 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.750583 (+ 860us) server_negotiation.cc:407] Connection header received
0619 06:51:15.750896 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.750902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.750984 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.751080 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:15.751657 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.752313 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.754186 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.755181 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.755745 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.755816 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.756146 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.756272 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:51:15.756712 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.749637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:35483 (local address 127.18.39.66:37449)
0619 06:51:15.750477 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.750492 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.750672 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.751117 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.751120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.751142 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:15.751471 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.751478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.752455 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.752460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.754033 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.754044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.755787 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.755792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.755970 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.755994 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.756105 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.756238 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":212,"spinlock_wait_cycles":48768,"thread_start_us":401,"threads_started":1}
I20260619 06:51:15.756745 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.749634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41889 (local address 127.18.39.66:33093)
0619 06:51:15.750435 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:15.750455 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:15.750630 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:15.750994 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:15.750998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:15.751025 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:15.751438 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.751450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.752463 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.752468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.754031 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:15.754042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.755755 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.755762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:15.755964 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.755989 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:51:15.756065 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 06:51:15.756238 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":24,"thread_start_us":384,"threads_started":1}
I20260619 06:51:15.757855 25676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc6ea95a335439f889b5213f65bc67d" candidate_uuid: "46e1a52c023e41d2b2652bef95dc81ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e467617eed594e9a9611d0a3b1bdd644" is_pre_election: true
I20260619 06:51:15.757934 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc6ea95a335439f889b5213f65bc67d" candidate_uuid: "46e1a52c023e41d2b2652bef95dc81ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0698608120f452dacb282877229b4f6" is_pre_election: true
I20260619 06:51:15.758371 25676 raft_consensus.cc:2454] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46e1a52c023e41d2b2652bef95dc81ae in term 0.
I20260619 06:51:15.758384 25965 raft_consensus.cc:2454] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46e1a52c023e41d2b2652bef95dc81ae in term 0.
I20260619 06:51:15.759100 25759 leader_election.cc:302] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 46e1a52c023e41d2b2652bef95dc81ae, d0698608120f452dacb282877229b4f6, e467617eed594e9a9611d0a3b1bdd644; no voters:
I20260619 06:51:15.759511 26050 raft_consensus.cc:2781] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:15.759655 26050 raft_consensus.cc:482] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:15.759779 26050 raft_consensus.cc:3037] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:15.762184 26050 raft_consensus.cc:504] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.762778 26050 leader_election.cc:288] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [CANDIDATE]: Term 1 election: Requested vote from peers d0698608120f452dacb282877229b4f6 (127.18.39.67:35483), e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889)
I20260619 06:51:15.763334 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc6ea95a335439f889b5213f65bc67d" candidate_uuid: "46e1a52c023e41d2b2652bef95dc81ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0698608120f452dacb282877229b4f6"
I20260619 06:51:15.763340 25676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fc6ea95a335439f889b5213f65bc67d" candidate_uuid: "46e1a52c023e41d2b2652bef95dc81ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e467617eed594e9a9611d0a3b1bdd644"
I20260619 06:51:15.763552 25676 raft_consensus.cc:3037] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:15.763552 25965 raft_consensus.cc:3037] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:15.766800 25676 raft_consensus.cc:2454] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46e1a52c023e41d2b2652bef95dc81ae in term 1.
I20260619 06:51:15.766800 25965 raft_consensus.cc:2454] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46e1a52c023e41d2b2652bef95dc81ae in term 1.
I20260619 06:51:15.767244 25759 leader_election.cc:302] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46e1a52c023e41d2b2652bef95dc81ae, d0698608120f452dacb282877229b4f6; no voters:
I20260619 06:51:15.767565 26050 raft_consensus.cc:2781] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:15.768020 26050 raft_consensus.cc:686] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 LEADER]: Becoming Leader. State: Replica: 46e1a52c023e41d2b2652bef95dc81ae, State: Running, Role: LEADER
I20260619 06:51:15.768756 26050 consensus_queue.cc:237] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } }
I20260619 06:51:15.774695 24753 catalog_manager.cc:5495] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66). New cstate: current_term: 1 leader_uuid: "46e1a52c023e41d2b2652bef95dc81ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } health_report { overall_health: UNKNOWN } } }
I20260619 06:51:15.794656 26035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.788719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42960 (local address 127.18.39.66:43393)
0619 06:51:15.788878 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.788882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.788942 (+ 60us) server_negotiation.cc:407] Connection header received
0619 06:51:15.789130 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.789135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.789239 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.789416 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:15.789731 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.790826 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.792220 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.792858 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.792973 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.793441 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:15.794112 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:15.794263 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.794307 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.794387 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 06:51:15.803360 25676 raft_consensus.cc:1261] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Refusing update from remote peer 46e1a52c023e41d2b2652bef95dc81ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:15.803360 25965 raft_consensus.cc:1261] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Refusing update from remote peer 46e1a52c023e41d2b2652bef95dc81ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:15.804504 26053 consensus_queue.cc:1035] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:15.804908 26050 consensus_queue.cc:1035] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:15.817226 26057 mvcc.cc:205] Tried to move back new op lower bound from 7298465283279196160 to 7298465283152728064. Current Snapshot: MvccSnapshot[applied={T|T < 7298465283279196160}]
I20260619 06:51:15.819482 26059 mvcc.cc:205] Tried to move back new op lower bound from 7298465283279196160 to 7298465283152728064. Current Snapshot: MvccSnapshot[applied={T|T < 7298465283279196160}]
I20260619 06:51:15.840250 24751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:15.840480 24751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:15.847393 24751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7bcafeefff94aedad77a5a8a483df88: Sending DeleteTablet for 3 replicas of tablet 1fc6ea95a335439f889b5213f65bc67d
I20260619 06:51:15.848560 25945 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fc6ea95a335439f889b5213f65bc67d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:15 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:52602
I20260619 06:51:15.849169 25656 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fc6ea95a335439f889b5213f65bc67d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:15 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:40836
I20260619 06:51:15.849216 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fc6ea95a335439f889b5213f65bc67d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:15 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42958
I20260619 06:51:15.850683 26066 tablet_replica.cc:330] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: stopping tablet replica
I20260619 06:51:15.851223 26066 raft_consensus.cc:2229] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:15.851841 26066 raft_consensus.cc:2258] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:15.854403 26066 ts_tablet_manager.cc:1905] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:15.863855 26066 ts_tablet_manager.cc:1918] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:15.864158 26066 log.cc:1196] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/1fc6ea95a335439f889b5213f65bc67d
I20260619 06:51:15.864774 26066 ts_tablet_manager.cc:1939] T 1fc6ea95a335439f889b5213f65bc67d P d0698608120f452dacb282877229b4f6: Deleting consensus metadata
I20260619 06:51:15.866863 26068 tablet_replica.cc:330] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: stopping tablet replica
I20260619 06:51:15.867125 24739 catalog_manager.cc:4841] TS d0698608120f452dacb282877229b4f6 (127.18.39.67:35483): tablet 1fc6ea95a335439f889b5213f65bc67d (table test-table [id=9e657be818734d9b8f975befd83855fc]) successfully deleted
I20260619 06:51:15.867450 26068 raft_consensus.cc:2229] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:15.867956 26068 raft_consensus.cc:2258] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:15.870121 26068 ts_tablet_manager.cc:1905] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:15.873633 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 24721
I20260619 06:51:15.874904 24783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:15.879593 26067 tablet_replica.cc:330] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: stopping tablet replica
I20260619 06:51:15.880137 26067 raft_consensus.cc:2229] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:15.880746 26067 raft_consensus.cc:2258] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:15.882720 26067 ts_tablet_manager.cc:1905] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:15.886654 26068 ts_tablet_manager.cc:1918] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:15.886831 26068 log.cc:1196] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/1fc6ea95a335439f889b5213f65bc67d
I20260619 06:51:15.887421 26068 ts_tablet_manager.cc:1939] T 1fc6ea95a335439f889b5213f65bc67d P e467617eed594e9a9611d0a3b1bdd644: Deleting consensus metadata
I20260619 06:51:15.890834 26067 ts_tablet_manager.cc:1918] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:15.891048 26067 log.cc:1196] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/1fc6ea95a335439f889b5213f65bc67d
I20260619 06:51:15.891716 26067 ts_tablet_manager.cc:1939] T 1fc6ea95a335439f889b5213f65bc67d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting consensus metadata
I20260619 06:51:15.899938 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.852024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.18.39.125:34959)
0619 06:51:15.860089 (+ 8065us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.860095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.871915 (+ 11820us) server_negotiation.cc:407] Connection header received
0619 06:51:15.872156 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.872163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.872259 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.872388 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.879081 (+ 6693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.880223 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.883199 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.884280 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.884388 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.892784 (+ 8396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.892794 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.892801 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.892851 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.896549 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.897501 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.897508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.897512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.897671 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.898202 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.898207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.898210 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.898576 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.898812 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.899372 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.899588 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7660,"thread_start_us":7531,"threads_started":1}
I20260619 06:51:15.919090 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:15.872752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.18.39.124:44217)
0619 06:51:15.894218 (+ 21466us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:15.894226 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:15.901318 (+ 7092us) server_negotiation.cc:407] Connection header received
0619 06:51:15.901607 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:15.901613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:15.901694 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:15.901780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:15.903764 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.904974 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.907327 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:15.907956 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:15.908097 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:15.912251 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:15.912259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:15.912264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:15.912309 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:15.915944 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.916825 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.916831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.916835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.916992 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:15.917438 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:15.917445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:15.917451 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:15.917871 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:15.918125 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:15.918460 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 06:51:15.918698 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21315,"thread_start_us":21191,"threads_started":1}
I20260619 06:51:16.013031 24739 catalog_manager.cc:4841] TS 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393): tablet 1fc6ea95a335439f889b5213f65bc67d (table test-table [id=9e657be818734d9b8f975befd83855fc]) successfully deleted
I20260619 06:51:16.016469 24738 catalog_manager.cc:4841] TS e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889): tablet 1fc6ea95a335439f889b5213f65bc67d (table test-table [id=9e657be818734d9b8f975befd83855fc]) successfully deleted
I20260619 06:51:16.017032 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 24721
I20260619 06:51:16.059576 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:16.043617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32938 (local address 127.18.39.125:34959)
0619 06:51:16.043812 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:16.043818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:16.043846 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:51:16.044156 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:16.044164 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:16.044257 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:16.044337 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:16.046478 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:16.047364 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:16.049171 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:16.049664 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:16.049802 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:16.052990 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:16.052996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:16.053002 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:16.053039 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:16.056459 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:16.057263 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:16.057268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:16.057272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:16.057442 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:16.057846 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:16.057851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:16.057854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:16.058215 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:16.058475 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:16.058904 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 06:51:16.059124 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:51:16.061141 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:16.044440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47834 (local address 127.18.39.124:44217)
0619 06:51:16.044618 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:16.044625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:16.044880 (+ 255us) server_negotiation.cc:407] Connection header received
0619 06:51:16.045131 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:16.045137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:16.045231 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:16.045295 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:16.047645 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:16.048727 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:16.050647 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:16.051312 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:16.051456 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:16.054570 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:16.054577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:16.054583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:16.054627 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:16.058082 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:16.058852 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:16.058856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:16.058860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:16.059029 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:16.059413 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:16.059418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:16.059430 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:16.059802 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:16.060030 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:16.060590 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 06:51:16.060806 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:51:16.826356 26079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:16.825629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:46767)
0619 06:51:16.826022 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:16.826129 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":208,"threads_started":1}
W20260619 06:51:16.827234 26010 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:40475 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:16.832161 26080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:16.831484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:58775)
0619 06:51:16.831915 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:16.832017 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":210,"threads_started":1}
W20260619 06:51:16.832516 25869 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:40475 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:16.835187 26081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:16.834459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:55989)
0619 06:51:16.834917 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:16.835053 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":248,"threads_started":1}
W20260619 06:51:16.835572 25721 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:40475 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:17.348564 26082 raft_consensus.cc:482] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bcafeefff94aedad77a5a8a483df88)
I20260619 06:51:17.348803 26083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7bcafeefff94aedad77a5a8a483df88)
I20260619 06:51:17.348907 26082 raft_consensus.cc:504] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } }
I20260619 06:51:17.349077 26083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } }
I20260619 06:51:17.350436 26082 leader_election.cc:288] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475), 6de890eb67d249a993a29c8289bbf943 (127.18.39.124:44217)
I20260619 06:51:17.350472 26083 leader_election.cc:288] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475), baf88b00f08b497f9f1decf35a146f1b (127.18.39.125:34959)
I20260619 06:51:17.352761 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.350649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34212)
0619 06:51:17.352260 (+ 1611us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.352429 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1486,"thread_start_us":1364,"threads_started":1}
I20260619 06:51:17.353456 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.350571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34200)
0619 06:51:17.352961 (+ 2390us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.353123 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2267,"thread_start_us":2122,"threads_started":1}
W20260619 06:51:17.353866 25340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
W20260619 06:51:17.354473 25527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:17.358733 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.358245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34232)
0619 06:51:17.358378 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.358507 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 06:51:17.358976 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.358339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34236)
0619 06:51:17.358500 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.358721 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 06:51:17.359066 25527 leader_election.cc:334] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475): Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
W20260619 06:51:17.359575 25340 leader_election.cc:334] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475): Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:17.362404 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.352439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:44217 (local address 127.0.0.1:47844)
0619 06:51:17.352984 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.353008 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.353179 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.354882 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.354890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.354931 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.355260 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.355276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.357131 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.357136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.359394 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.359444 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.361576 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.361582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.361761 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.361791 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.361897 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.362035 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":922,"threads_started":1}
I20260619 06:51:17.362427 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.353964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.18.39.124:44217)
0619 06:51:17.354524 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.354529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.354550 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:51:17.354622 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.354629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.354698 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.355444 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.355698 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.356977 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.360268 (+ 3291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.361380 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.361566 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.361672 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.361920 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.362045 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":279,"threads_started":1}
I20260619 06:51:17.363297 25552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "baf88b00f08b497f9f1decf35a146f1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6de890eb67d249a993a29c8289bbf943" is_pre_election: true
I20260619 06:51:17.363628 25552 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate baf88b00f08b497f9f1decf35a146f1b in term 1.
I20260619 06:51:17.364154 25340 leader_election.cc:302] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6de890eb67d249a993a29c8289bbf943, baf88b00f08b497f9f1decf35a146f1b; no voters: c7bcafeefff94aedad77a5a8a483df88
I20260619 06:51:17.364512 26082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 06:51:17.364635 26082 raft_consensus.cc:482] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7bcafeefff94aedad77a5a8a483df88)
I20260619 06:51:17.364713 26084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.353148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:34959 (local address 127.0.0.1:32954)
0619 06:51:17.353721 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.353746 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.353898 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.358645 (+ 4747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.358654 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.358687 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.359057 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.359071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.360972 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.360976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.362702 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.362715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.363966 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.363972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.364153 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.364192 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.364320 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.364457 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":570,"threads_started":1}
I20260619 06:51:17.364737 26082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:51:17.364979 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.354614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.18.39.125:34959)
0619 06:51:17.358311 (+ 3697us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.358317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.358336 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:17.358391 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.358396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.358464 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.359350 (+ 886us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.359700 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.360814 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.362867 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.363775 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.363907 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.364029 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.364414 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.364560 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3566,"thread_start_us":3461,"threads_started":1}
I20260619 06:51:17.365471 25364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6de890eb67d249a993a29c8289bbf943" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "baf88b00f08b497f9f1decf35a146f1b" is_pre_election: true
I20260619 06:51:17.367223 26082 raft_consensus.cc:504] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } }
I20260619 06:51:17.367748 25364 raft_consensus.cc:2379] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6de890eb67d249a993a29c8289bbf943 in current term 2: Already voted for candidate baf88b00f08b497f9f1decf35a146f1b in this term.
I20260619 06:51:17.368052 26082 leader_election.cc:288] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 election: Requested vote from peers c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475), 6de890eb67d249a993a29c8289bbf943 (127.18.39.124:44217)
I20260619 06:51:17.368598 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.368207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34250)
0619 06:51:17.368345 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.368446 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 06:51:17.368831 25552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "baf88b00f08b497f9f1decf35a146f1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6de890eb67d249a993a29c8289bbf943"
I20260619 06:51:17.368932 25527 leader_election.cc:302] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6de890eb67d249a993a29c8289bbf943; no voters: baf88b00f08b497f9f1decf35a146f1b, c7bcafeefff94aedad77a5a8a483df88
I20260619 06:51:17.369091 25552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:51:17.371398 25552 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate baf88b00f08b497f9f1decf35a146f1b in term 2.
I20260619 06:51:17.371637 26083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:51:17.371994 25340 leader_election.cc:302] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de890eb67d249a993a29c8289bbf943, baf88b00f08b497f9f1decf35a146f1b; no voters:
I20260619 06:51:17.372320 26082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:51:17.372865 26082 raft_consensus.cc:686] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [term 2 LEADER]: Becoming Leader. State: Replica: baf88b00f08b497f9f1decf35a146f1b, State: Running, Role: LEADER
I20260619 06:51:17.372990 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.372395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:34252)
0619 06:51:17.372618 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.372754 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260619 06:51:17.373140 25340 leader_election.cc:334] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475): Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
I20260619 06:51:17.373492 26082 consensus_queue.cc:237] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } }
I20260619 06:51:17.375546 26092 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader baf88b00f08b497f9f1decf35a146f1b. Latest consensus state: current_term: 2 leader_uuid: "baf88b00f08b497f9f1decf35a146f1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:17.375960 26092 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:17.376602 26094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:17.380226 26094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:17.381058 26094 catalog_manager.cc:1259] Loaded cluster ID: bc687f4e85d147699224912d02287b09
I20260619 06:51:17.381158 26094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:17.382347 26094 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:17.383140 26094 catalog_manager.cc:5879] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Loaded TSK: 0
I20260619 06:51:17.384032 26094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:51:17.418208 25353 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:17.426168 25353 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:17.430841 25552 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [term 2 FOLLOWER]: Refusing update from remote peer baf88b00f08b497f9f1decf35a146f1b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 06:51:17.430909 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.430402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:58810)
0619 06:51:17.430629 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.430729 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 06:51:17.431694 26092 consensus_queue.cc:1035] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 06:51:17.432173 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.431764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:58814)
0619 06:51:17.431921 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.432024 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 06:51:17.432396 25340 consensus_peers.cc:487] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b -> Peer c7bcafeefff94aedad77a5a8a483df88 (127.18.39.126:40475): Couldn't send request to peer c7bcafeefff94aedad77a5a8a483df88. Status: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 06:51:17.434793 26083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baf88b00f08b497f9f1decf35a146f1b. Latest consensus state: current_term: 2 leader_uuid: "baf88b00f08b497f9f1decf35a146f1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:17.435130 26083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:17.436615 26092 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "baf88b00f08b497f9f1decf35a146f1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:17.436944 26092 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:17.437426 26083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "baf88b00f08b497f9f1decf35a146f1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c7bcafeefff94aedad77a5a8a483df88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40475 } } peers { permanent_uuid: "baf88b00f08b497f9f1decf35a146f1b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 34959 } attrs { promote: false } } peers { permanent_uuid: "6de890eb67d249a993a29c8289bbf943" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 44217 } attrs { promote: false } } }
I20260619 06:51:17.437722 26083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6de890eb67d249a993a29c8289bbf943 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:51:17.455224 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.448126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.18.39.67:35483)
0619 06:51:17.448614 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.448619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.448634 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:51:17.448691 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.448695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.448777 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.448883 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.449767 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.451109 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.453152 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.454199 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.454369 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.454470 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.454773 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.454911 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":271,"threads_started":1}
I20260619 06:51:17.455262 26102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.447723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:35483 (local address 127.0.0.1:33696)
0619 06:51:17.448248 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.448266 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.448411 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.448937 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.448945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.448981 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.449525 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.449541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.451285 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.451290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.452769 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.452784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.454405 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.454411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.454605 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.454630 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.454747 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.454901 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":562,"threads_started":1}
I20260619 06:51:17.456029 25945 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (DEFAULT_TABLE table=test-table [id=7472157a527a44e0ae37d9127165826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:17.456506 25945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:17.462952 26105 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Bootstrap starting.
I20260619 06:51:17.464067 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.446926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:41889 (local address 127.0.0.1:56842)
0619 06:51:17.447476 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.447497 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.447622 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.458793 (+ 11171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.458797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.458822 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.458952 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.458959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.460304 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.460308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.461822 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.461835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.463344 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.463351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.463546 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.463582 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.463649 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.463785 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408}
I20260619 06:51:17.464787 26105 tablet_bootstrap.cc:654] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:17.464942 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.447130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.18.39.65:41889)
0619 06:51:17.458481 (+ 11351us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.458489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.458508 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:17.458572 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.458577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.458651 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.458758 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.459115 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.460152 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.461972 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.463167 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.464301 (+ 1134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.464444 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.464499 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.464640 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11232,"thread_start_us":11156,"threads_started":1}
I20260619 06:51:17.465939 25656 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (DEFAULT_TABLE table=test-table [id=7472157a527a44e0ae37d9127165826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:17.466498 25656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:17.468012 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.447858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.18.39.66:43393)
0619 06:51:17.457744 (+ 9886us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.457751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.457771 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:51:17.457844 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.457850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.457927 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.458032 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.463094 (+ 5062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.464105 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.466174 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.467107 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.467248 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.467353 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.467608 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.467736 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9751,"thread_start_us":9646,"threads_started":1}
I20260619 06:51:17.468133 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.447390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43393 (local address 127.0.0.1:59996)
0619 06:51:17.447550 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.447569 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.447687 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.462545 (+ 14858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.462550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.462586 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.462891 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.462904 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.464277 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.464283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.466024 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.466035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.467297 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.467303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.467455 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.467477 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.467581 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.467726 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 06:51:17.471072 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (DEFAULT_TABLE table=test-table [id=7472157a527a44e0ae37d9127165826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:17.471590 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:17.472180 26105 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: No bootstrap required, opened a new log
I20260619 06:51:17.472335 26105 ts_tablet_manager.cc:1397] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 06:51:17.473182 26105 raft_consensus.cc:348] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.473448 26105 raft_consensus.cc:374] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:17.473547 26105 raft_consensus.cc:729] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0698608120f452dacb282877229b4f6, State: Initialized, Role: FOLLOWER
I20260619 06:51:17.473816 26105 consensus_queue.cc:260] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.479718 26109 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Bootstrap starting.
I20260619 06:51:17.481884 26107 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Bootstrap starting.
I20260619 06:51:17.482321 26105 ts_tablet_manager.cc:1428] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20260619 06:51:17.483855 26107 tablet_bootstrap.cc:654] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:17.484005 26109 tablet_bootstrap.cc:654] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:17.486654 26107 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: No bootstrap required, opened a new log
I20260619 06:51:17.486838 26107 ts_tablet_manager.cc:1397] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 06:51:17.487897 26107 raft_consensus.cc:348] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.488219 26107 raft_consensus.cc:374] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:17.488354 26107 raft_consensus.cc:729] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e467617eed594e9a9611d0a3b1bdd644, State: Initialized, Role: FOLLOWER
I20260619 06:51:17.488669 26107 consensus_queue.cc:260] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.489758 26107 ts_tablet_manager.cc:1428] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:17.494614 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.482695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:52629)
0619 06:51:17.493975 (+ 11280us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.494242 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11066,"thread_start_us":11233,"threads_started":1}
I20260619 06:51:17.495864 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.495367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:32949)
0619 06:51:17.495574 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.495672 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260619 06:51:17.497661 26012 heartbeater.cc:503] Master 127.18.39.125:34959 was elected leader, sending a full tablet report...
I20260619 06:51:17.499991 26109 tablet_bootstrap.cc:492] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: No bootstrap required, opened a new log
I20260619 06:51:17.500161 26109 ts_tablet_manager.cc:1397] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.011s
I20260619 06:51:17.500881 26109 raft_consensus.cc:348] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.501070 26109 raft_consensus.cc:374] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:17.501188 26109 raft_consensus.cc:729] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e1a52c023e41d2b2652bef95dc81ae, State: Initialized, Role: FOLLOWER
I20260619 06:51:17.501447 26109 consensus_queue.cc:260] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.502194 26109 ts_tablet_manager.cc:1428] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 06:51:17.502699 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.490006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:54385)
0619 06:51:17.501921 (+ 11915us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.502096 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11744,"threads_started":1}
I20260619 06:51:17.504523 25723 heartbeater.cc:503] Master 127.18.39.125:34959 was elected leader, sending a full tablet report...
I20260619 06:51:17.504729 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.503638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:36337)
0619 06:51:17.504478 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.504546 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":745}
I20260619 06:51:17.504966 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.502532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:46557)
0619 06:51:17.504010 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.504156 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1336,"thread_start_us":1217,"threads_started":1}
I20260619 06:51:17.506250 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.505894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:53423)
0619 06:51:17.506017 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.506099 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 06:51:17.506852 25871 heartbeater.cc:503] Master 127.18.39.125:34959 was elected leader, sending a full tablet report...
I20260619 06:51:17.512444 26113 raft_consensus.cc:482] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:17.512681 26113 raft_consensus.cc:504] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.514178 26113 leader_election.cc:288] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393), d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:17.521536 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.514490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:59783 (local address 127.18.39.66:43393)
0619 06:51:17.514690 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.514697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.514715 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:17.514859 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.514867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.514961 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.515064 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.515737 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.516752 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.519265 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.520487 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.520659 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.520738 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.521123 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.521222 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:51:17.521790 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.514982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:40941 (local address 127.18.39.67:35483)
0619 06:51:17.515217 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.515223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.515397 (+ 174us) server_negotiation.cc:407] Connection header received
0619 06:51:17.515602 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.515607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.515695 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.515786 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:17.516239 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.517161 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.519469 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.520695 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.520838 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.520888 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.521301 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.521454 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:51:17.521874 26114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.514360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43393 (local address 127.18.39.65:59783)
0619 06:51:17.514544 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.514570 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.514716 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.515247 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.515250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.515297 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.515556 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.515563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.516945 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.516952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.519132 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.519146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.520676 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.520681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.520931 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.520970 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.521082 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.521244 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260619 06:51:17.521898 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.514690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:35483 (local address 127.18.39.65:40941)
0619 06:51:17.515303 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.515316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:17.515456 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:17.515803 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:17.515807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:17.515831 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:17.516104 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.516111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.517265 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.517270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.519338 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:17.519347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.520953 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.520959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:17.521151 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.521182 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 06:51:17.521262 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 06:51:17.521407 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":332,"threads_started":1}
I20260619 06:51:17.522578 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb8a039be1c444cb9e2813cfdcbdc3d" candidate_uuid: "e467617eed594e9a9611d0a3b1bdd644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0698608120f452dacb282877229b4f6" is_pre_election: true
I20260619 06:51:17.522826 25965 raft_consensus.cc:2454] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e467617eed594e9a9611d0a3b1bdd644 in term 0.
I20260619 06:51:17.522768 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb8a039be1c444cb9e2813cfdcbdc3d" candidate_uuid: "e467617eed594e9a9611d0a3b1bdd644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e1a52c023e41d2b2652bef95dc81ae" is_pre_election: true
I20260619 06:51:17.523216 25824 raft_consensus.cc:2454] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e467617eed594e9a9611d0a3b1bdd644 in term 0.
I20260619 06:51:17.523411 25611 leader_election.cc:302] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0698608120f452dacb282877229b4f6, e467617eed594e9a9611d0a3b1bdd644; no voters:
I20260619 06:51:17.523809 26113 raft_consensus.cc:2781] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:17.523909 26113 raft_consensus.cc:482] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:17.524053 26113 raft_consensus.cc:3037] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:17.526521 26113 raft_consensus.cc:504] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.527206 26113 leader_election.cc:288] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [CANDIDATE]: Term 1 election: Requested vote from peers 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393), d0698608120f452dacb282877229b4f6 (127.18.39.67:35483)
I20260619 06:51:17.527648 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb8a039be1c444cb9e2813cfdcbdc3d" candidate_uuid: "e467617eed594e9a9611d0a3b1bdd644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46e1a52c023e41d2b2652bef95dc81ae"
I20260619 06:51:17.527887 25824 raft_consensus.cc:3037] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:17.528540 25965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb8a039be1c444cb9e2813cfdcbdc3d" candidate_uuid: "e467617eed594e9a9611d0a3b1bdd644" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0698608120f452dacb282877229b4f6"
I20260619 06:51:17.528753 25965 raft_consensus.cc:3037] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:17.530398 25824 raft_consensus.cc:2454] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e467617eed594e9a9611d0a3b1bdd644 in term 1.
I20260619 06:51:17.531613 25965 raft_consensus.cc:2454] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e467617eed594e9a9611d0a3b1bdd644 in term 1.
I20260619 06:51:17.532080 25611 leader_election.cc:302] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0698608120f452dacb282877229b4f6, e467617eed594e9a9611d0a3b1bdd644; no voters:
I20260619 06:51:17.532371 26113 raft_consensus.cc:2781] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:17.532862 26113 raft_consensus.cc:686] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 1 LEADER]: Becoming Leader. State: Replica: e467617eed594e9a9611d0a3b1bdd644, State: Running, Role: LEADER
I20260619 06:51:17.533282 26113 consensus_queue.cc:237] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } }
I20260619 06:51:17.533939 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.533433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:42273)
0619 06:51:17.533663 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.533782 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 06:51:17.537268 25354 catalog_manager.cc:5495] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 reported cstate change: term changed from 0 to 1, leader changed from <none> to e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "e467617eed594e9a9611d0a3b1bdd644" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467617eed594e9a9611d0a3b1bdd644" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 41889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 } health_report { overall_health: UNKNOWN } } }
I20260619 06:51:17.558528 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.553564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.18.39.65:41889)
0619 06:51:17.553744 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.553748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.553766 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:17.553936 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.553939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.554022 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.554144 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:17.554462 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.555145 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.556396 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.556855 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.556956 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.557311 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:17.557972 (+ 661us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:17.558084 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.558163 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.558228 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 06:51:17.562981 25965 raft_consensus.cc:1261] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Refusing update from remote peer e467617eed594e9a9611d0a3b1bdd644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:17.563225 25824 raft_consensus.cc:1261] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Refusing update from remote peer e467617eed594e9a9611d0a3b1bdd644: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:17.563774 26113 consensus_queue.cc:1035] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0698608120f452dacb282877229b4f6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 35483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:17.564142 26118 consensus_queue.cc:1035] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46e1a52c023e41d2b2652bef95dc81ae" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:17.566443 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.565907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:35361)
0619 06:51:17.566083 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.566212 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 06:51:17.567242 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.566653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:41937)
0619 06:51:17.566875 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.567019 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 06:51:17.579631 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.578849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:49433)
0619 06:51:17.579310 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.579433 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312}
I20260619 06:51:17.579631 26110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.579010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.67:48243)
0619 06:51:17.579340 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.579444 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260619 06:51:17.581362 26121 mvcc.cc:205] Tried to move back new op lower bound from 7298465290490052608 to 7298465290381066240. Current Snapshot: MvccSnapshot[applied={T|T < 7298465290490052608}]
I20260619 06:51:17.581725 26117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.580543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.65:44463)
0619 06:51:17.581475 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.581573 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":820}
I20260619 06:51:17.583292 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.582808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.18.39.66:50295)
0619 06:51:17.583025 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.583121 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260619 06:51:17.594565 25353 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:17.594864 25353 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:17.600812 25353 catalog_manager.cc:5782] T 00000000000000000000000000000000 P baf88b00f08b497f9f1decf35a146f1b: Sending DeleteTablet for 3 replicas of tablet 7cb8a039be1c444cb9e2813cfdcbdc3d
I20260619 06:51:17.601907 25656 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:17 UTC) from {username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:56842
I20260619 06:51:17.602160 25945 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:17 UTC) from {username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:33696
I20260619 06:51:17.602377 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb8a039be1c444cb9e2813cfdcbdc3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:17 UTC) from {username='oryx', principal='oryx/127.18.39.125@KRBTEST.COM'} at 127.0.0.1:59996
I20260619 06:51:17.603906 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25529
I20260619 06:51:17.618527 26132 tablet_replica.cc:330] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: stopping tablet replica
I20260619 06:51:17.618789 26132 raft_consensus.cc:2229] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:17.619146 26132 raft_consensus.cc:2258] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:17.620693 26131 tablet_replica.cc:330] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: stopping tablet replica
I20260619 06:51:17.620923 26131 raft_consensus.cc:2229] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:17.621322 26131 raft_consensus.cc:2258] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:17.622645 26131 ts_tablet_manager.cc:1905] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:17.624373 26132 ts_tablet_manager.cc:1905] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:17.624955 26130 tablet_replica.cc:330] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: stopping tablet replica
I20260619 06:51:17.625684 26130 raft_consensus.cc:2229] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:17.629974 26130 raft_consensus.cc:2258] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:17.631328 26131 ts_tablet_manager.cc:1918] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:17.631461 26131 log.cc:1196] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/7cb8a039be1c444cb9e2813cfdcbdc3d
I20260619 06:51:17.631835 26130 ts_tablet_manager.cc:1905] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:17.631929 26131 ts_tablet_manager.cc:1939] T 7cb8a039be1c444cb9e2813cfdcbdc3d P d0698608120f452dacb282877229b4f6: Deleting consensus metadata
I20260619 06:51:17.633179 25340 catalog_manager.cc:4841] TS d0698608120f452dacb282877229b4f6 (127.18.39.67:35483): tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (table test-table [id=7472157a527a44e0ae37d9127165826a]) successfully deleted
I20260619 06:51:17.637080 26130 ts_tablet_manager.cc:1918] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:17.637255 26130 log.cc:1196] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/7cb8a039be1c444cb9e2813cfdcbdc3d
I20260619 06:51:17.637822 26130 ts_tablet_manager.cc:1939] T 7cb8a039be1c444cb9e2813cfdcbdc3d P e467617eed594e9a9611d0a3b1bdd644: Deleting consensus metadata
I20260619 06:51:17.639045 25339 catalog_manager.cc:4841] TS e467617eed594e9a9611d0a3b1bdd644 (127.18.39.65:41889): tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (table test-table [id=7472157a527a44e0ae37d9127165826a]) successfully deleted
I20260619 06:51:17.641148 26132 ts_tablet_manager.cc:1918] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:17.641309 26132 log.cc:1196] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/7cb8a039be1c444cb9e2813cfdcbdc3d
I20260619 06:51:17.641794 26132 ts_tablet_manager.cc:1939] T 7cb8a039be1c444cb9e2813cfdcbdc3d P 46e1a52c023e41d2b2652bef95dc81ae: Deleting consensus metadata
I20260619 06:51:17.642900 25340 catalog_manager.cc:4841] TS 46e1a52c023e41d2b2652bef95dc81ae (127.18.39.66:43393): tablet 7cb8a039be1c444cb9e2813cfdcbdc3d (table test-table [id=7472157a527a44e0ae37d9127165826a]) successfully deleted
I20260619 06:51:17.644484 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.605571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.18.39.65:41889)
0619 06:51:17.605796 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.605805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.619790 (+ 13985us) server_negotiation.cc:407] Connection header received
0619 06:51:17.620044 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.620054 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.620193 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.620324 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:17.625530 (+ 5206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.626757 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.631064 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.631816 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.631940 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.637537 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:17.637561 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:17.637586 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:17.637637 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:17.641231 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.642148 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.642158 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.642161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.642304 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.642706 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.642712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.642715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.643232 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:17.643592 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.643881 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.644116 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:51:17.645227 25716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:17.750159 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25529
I20260619 06:51:17.768875 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25733
I20260619 06:51:17.782555 26103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.769647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.18.39.66:43393)
0619 06:51:17.769831 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.769837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.769934 (+ 97us) server_negotiation.cc:407] Connection header received
0619 06:51:17.770150 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.770156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.770229 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.770343 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:17.771601 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.772328 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.773774 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.774185 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.774323 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.776870 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:17.776893 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:17.776910 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:17.776941 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:17.779729 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.780422 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.780431 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.780435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.780572 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.780941 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.780948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.780951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.781315 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:17.781643 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.781966 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.782179 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:51:17.783216 25864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:17.909600 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25733
I20260619 06:51:17.930871 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25882
I20260619 06:51:17.944797 26104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.931626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33708 (local address 127.18.39.67:35483)
0619 06:51:17.931861 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:17.931866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:17.931885 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:17.932023 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:17.932028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:17.932088 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:17.932171 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:17.933578 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.934304 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.935487 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:17.935869 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:17.935961 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:17.939162 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:17.939178 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:17.939194 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:17.939242 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:17.942252 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.942929 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.942936 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.942938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.943067 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:17.943414 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:17.943417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:17.943419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:17.943791 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:17.943996 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:17.944257 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 06:51:17.944414 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260619 06:51:17.945564 26005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:17.948397 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.947995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:58828)
0619 06:51:17.948142 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.948242 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 06:51:17.950044 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:17.949688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40475 (local address 127.0.0.1:58842)
0619 06:51:17.949856 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:17.949927 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:40475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 06:51:18.041419 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25882
I20260619 06:51:18.058292 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25215
I20260619 06:51:18.059473 25384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:18.181677 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25215
I20260619 06:51:18.202266 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 25316
I20260619 06:51:18.203294 25572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:18.317946 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 25316
2026-06-19T06:51:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9852 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:18 dist-test-slave-1h5s krb5kdc[26142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:18 dist-test-slave-1h5s krb5kdc[26142](info): set up 2 sockets
Jun 19 06:51:18 dist-test-slave-1h5s krb5kdc[26142](info): commencing operation
krb5kdc: starting...
W20260619 06:51:19.426290 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:19 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab.
Entry for principal oryx/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab.
I20260619 06:51:19.646931 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42837
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab
--principal=oryx/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:42837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:19.969451 26155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:19.969759 26155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:19.969939 26155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:19.977200 26155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:19.977351 26155 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:51:19.977444 26155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:19.977550 26155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:19.977631 26155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:19.977720 26155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:19.987612 26155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:42837
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.126.keytab
--principal=oryx/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:19.988929 26155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:19.990276 26155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:20.001921 26163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:20.001840 26161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:20.001888 26160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:20.003209 26155 server_base.cc:1034] running on GCE node
I20260619 06:51:20.004098 26155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:20.005643 26155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:20.006901 26155 hybrid_clock.cc:648] HybridClock initialized: now 1781851880006818 us; error 59 us; skew 500 ppm
I20260619 06:51:20.007233 26155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:20 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851880, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:20.010406 26155 init.cc:377] Logged in from keytab as oryx/127.18.39.126@KRBTEST.COM (short username oryx)
I20260619 06:51:20.012039 26155 webserver.cc:460] Webserver started at http://127.18.39.126:35197/ using document root <none> and password file <none>
I20260619 06:51:20.012770 26155 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:20.012907 26155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:20.013227 26155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:20.016305 26155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "1d6cebc734e14bb396052f947becc755"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "5bda3e08ce6ac405858a62519fce0e42"
server_key_iv: "8df6c38990d599e1b93c18732541d205"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.017127 26155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "1d6cebc734e14bb396052f947becc755"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "5bda3e08ce6ac405858a62519fce0e42"
server_key_iv: "8df6c38990d599e1b93c18732541d205"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.022471 26155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:51:20.026662 26170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:20.028317 26155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:51:20.028537 26155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "1d6cebc734e14bb396052f947becc755"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "5bda3e08ce6ac405858a62519fce0e42"
server_key_iv: "8df6c38990d599e1b93c18732541d205"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.028811 26155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:20.057425 26155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:20.061619 26155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:20.061853 26155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:20.083604 26155 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:42837
I20260619 06:51:20.083624 26231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:42837 every 8 connection(s)
I20260619 06:51:20.085038 26155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:20.087828 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26155
I20260619 06:51:20.088050 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:20.088891 18589 external_mini_cluster.cc:1428] Setting key 71f01422e440ee2fafa0487bb5e42468
I20260619 06:51:20.089709 26232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:20.104429 26232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Bootstrap starting.
Jun 19 06:51:20 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:20.108414 26232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:20.109851 26232 log.cc:824] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:20.113382 26232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: No bootstrap required, opened a new log
I20260619 06:51:20.114885 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:20.091055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.18.39.126:42837)
0619 06:51:20.091889 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:20.091901 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:20.091942 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:51:20.094340 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:20.094365 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:20.094828 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:20.095242 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:20.097126 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.098559 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.100359 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.101096 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.101251 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:20.105936 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:20.105949 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:20.105969 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:20.106033 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:20.110053 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.111078 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.111085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.111091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.111366 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.111730 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.111734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.111737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.112390 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:20.112680 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:20.113055 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 06:51:20.113525 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":415,"threads_started":1}
I20260619 06:51:20.121892 26232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } }
I20260619 06:51:20.122332 26232 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:20.122430 26232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d6cebc734e14bb396052f947becc755, State: Initialized, Role: FOLLOWER
I20260619 06:51:20.123036 26232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } }
I20260619 06:51:20.123256 26232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:20.123409 26232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:20.123579 26232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:20.125797 26232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } }
I20260619 06:51:20.126324 26232 leader_election.cc:302] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d6cebc734e14bb396052f947becc755; no voters:
I20260619 06:51:20.126910 26232 leader_election.cc:288] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:20.127071 26237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:20.127868 26237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [term 1 LEADER]: Becoming Leader. State: Replica: 1d6cebc734e14bb396052f947becc755, State: Running, Role: LEADER
I20260619 06:51:20.128393 26237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } }
I20260619 06:51:20.129071 26232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:20.131405 26239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d6cebc734e14bb396052f947becc755. Latest consensus state: current_term: 1 leader_uuid: "1d6cebc734e14bb396052f947becc755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } } }
I20260619 06:51:20.131755 26239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:20.132247 26238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d6cebc734e14bb396052f947becc755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d6cebc734e14bb396052f947becc755" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42837 } } }
I20260619 06:51:20.132521 26238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:20.134793 26245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:20.140105 26245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:20.149880 26245 catalog_manager.cc:1347] Generated new cluster ID: d096997d85b9476fac2c565a2c160f5f
I20260619 06:51:20.150071 26245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:20.160662 26245 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:20.162599 26245 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:20.169930 26245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Generated new TSK 0
I20260619 06:51:20.170756 26245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
I20260619 06:51:20.235026 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42837
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:20.568905 26258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:20.569182 26258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:20.569424 26258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:20.576786 26258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:20.576910 26258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:20.577070 26258 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:51:20.577174 26258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:20.587519 26258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:42837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:20.588857 26258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:20.590269 26258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:20.603881 26263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:20.603787 26266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:20.603812 26264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:20.604641 26258 server_base.cc:1034] running on GCE node
I20260619 06:51:20.605257 26258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:20.606258 26258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:20.607527 26258 hybrid_clock.cc:648] HybridClock initialized: now 1781851880607457 us; error 49 us; skew 500 ppm
I20260619 06:51:20.607827 26258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:20 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851880, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:20.610997 26258 init.cc:377] Logged in from keytab as oryx/127.18.39.65@KRBTEST.COM (short username oryx)
I20260619 06:51:20.612525 26258 webserver.cc:460] Webserver started at http://127.18.39.65:33081/ using document root <none> and password file <none>
I20260619 06:51:20.613216 26258 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:20.613395 26258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:20.613808 26258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:20.616822 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "bb494569ce15479ab2d304c1e0d03028"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "e07242c00514cad8336308c7970a1d31"
server_key_iv: "7d67e1f294395c93dc61079b92242d21"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.617662 26258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "bb494569ce15479ab2d304c1e0d03028"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "e07242c00514cad8336308c7970a1d31"
server_key_iv: "7d67e1f294395c93dc61079b92242d21"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.622802 26258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260619 06:51:20.626483 26273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:20.627868 26258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:51:20.628057 26258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "bb494569ce15479ab2d304c1e0d03028"
format_stamp: "Formatted at 2026-06-19 06:51:20 on dist-test-slave-1h5s"
server_key: "e07242c00514cad8336308c7970a1d31"
server_key_iv: "7d67e1f294395c93dc61079b92242d21"
server_key_version: "encryptionkey@0"
I20260619 06:51:20.628290 26258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:20.652180 26258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:20.656133 26258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:20.656401 26258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:20.659179 26258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:20.659302 26258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:20.659480 26258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:20.659591 26258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 19 06:51:20 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851880, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:20.685886 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:20.659572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:38571 (local address 127.18.39.126:42837)
0619 06:51:20.660063 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:20.660072 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:20.661404 (+ 1332us) server_negotiation.cc:407] Connection header received
0619 06:51:20.663985 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:20.663990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:20.664094 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:20.664218 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:20.667537 (+ 3319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.668302 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.671636 (+ 3334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.672048 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.672149 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:20.677934 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:20.677944 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:20.677952 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:20.678004 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:20.681690 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.683020 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.683029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.683035 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.683197 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.683688 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.683697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.683703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.684172 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:20.684359 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:20.685065 (+ 706us) server_negotiation.cc:299] Negotiation successful
0619 06:51:20.685387 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":241,"threads_started":1}
I20260619 06:51:20.686867 26283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:20.660354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.65:38571)
0619 06:51:20.661188 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:20.661226 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:20.663696 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:20.664705 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:20.664722 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:20.666025 (+ 1303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:20.667289 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:20.667335 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.668507 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.668516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:20.671463 (+ 2947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:20.671484 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.672444 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.672464 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:20.672824 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:20.673864 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:20.673897 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:20.677630 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:20.681908 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:20.681923 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:20.681951 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:20.682793 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:20.683334 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:20.683339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:20.683342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:20.683558 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:20.684303 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:20.684309 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:20.684681 (+ 372us) client_negotiation.cc:769] Sending connection context
0619 06:51:20.684910 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 06:51:20.685319 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":352,"threads_started":1}
I20260619 06:51:20.708263 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:20.693484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:38061 (local address 127.18.39.126:42837)
0619 06:51:20.693691 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:20.693700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:20.694402 (+ 702us) server_negotiation.cc:407] Connection header received
0619 06:51:20.694628 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:20.694631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:20.694696 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:20.694773 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:20.696264 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.696987 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.699699 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.700229 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.700378 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:20.702589 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:20.702594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:20.702597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:20.702626 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:20.705126 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.705960 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.705966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.705970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.706133 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:20.706587 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:20.706593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:20.706599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:20.706957 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:20.707126 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:20.707535 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 06:51:20.707811 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:51:20.708442 26352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:20.693569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.65:38061)
0619 06:51:20.694280 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:20.694306 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:20.694496 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:20.694837 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:20.694844 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:20.695626 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:20.696100 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:20.696116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.697126 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.697134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:20.699535 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:20.699554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:20.700454 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:20.700474 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:20.700730 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:20.701631 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:20.701651 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:20.702402 (+ 751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:20.705262 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:20.705267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:20.705272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:20.705799 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:20.706270 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:20.706275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:20.706281 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:20.706456 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:20.707079 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:20.707082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:20.707244 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 06:51:20.707347 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:51:20.707732 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":337,"threads_started":1}
I20260619 06:51:20.715734 26258 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:45183
I20260619 06:51:20.715757 26393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:45183 every 8 connection(s)
I20260619 06:51:20.716986 26258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:20.719076 26394 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42837
I20260619 06:51:20.719307 26394 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:20.719959 26394 heartbeater.cc:511] Master 127.18.39.126:42837 requested a full tablet report, sending...
I20260619 06:51:20.722571 26187 ts_manager.cc:194] Registered new tserver with Master: bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183)
I20260619 06:51:20.724424 26187 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:38571
I20260619 06:51:20.727248 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26258
I20260619 06:51:20.727406 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:20.727995 18589 external_mini_cluster.cc:1428] Setting key ca5868ea2f3ee0f2194922edbd20371b
WARNING: no policy specified for oryx/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
I20260619 06:51:20.773975 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42837
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:21.098891 26398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:21.099210 26398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:21.099411 26398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:21.107196 26398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:21.107344 26398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:21.107517 26398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:51:21.107631 26398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:21.118021 26398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:42837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:21.119490 26398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:21.120903 26398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:21.134665 26403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:21.134593 26406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:21.134658 26404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:21.135778 26398 server_base.cc:1034] running on GCE node
I20260619 06:51:21.136425 26398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:21.137557 26398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:21.138804 26398 hybrid_clock.cc:648] HybridClock initialized: now 1781851881138728 us; error 57 us; skew 500 ppm
I20260619 06:51:21.139200 26398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:21.142508 26398 init.cc:377] Logged in from keytab as oryx/127.18.39.66@KRBTEST.COM (short username oryx)
I20260619 06:51:21.144023 26398 webserver.cc:460] Webserver started at http://127.18.39.66:38499/ using document root <none> and password file <none>
I20260619 06:51:21.144714 26398 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:21.144833 26398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:21.145150 26398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:21.148052 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "7e700c4a588d43139c79516c4622e495"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "976cf0dfc30a233befa9be942baeb8ee"
server_key_iv: "aa09d9562e00188abc71d32137a981e1"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.148833 26398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "7e700c4a588d43139c79516c4622e495"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "976cf0dfc30a233befa9be942baeb8ee"
server_key_iv: "aa09d9562e00188abc71d32137a981e1"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.153879 26398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 06:51:21.157147 26413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:21.158457 26398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:21.158641 26398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "7e700c4a588d43139c79516c4622e495"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "976cf0dfc30a233befa9be942baeb8ee"
server_key_iv: "aa09d9562e00188abc71d32137a981e1"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.158876 26398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:21.183112 26398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:21.187397 26398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:21.187673 26398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:21.190440 26398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:21.190539 26398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:21.190685 26398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:21.190773 26398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:21.215566 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.191011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:39915 (local address 127.18.39.126:42837)
0619 06:51:21.191203 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:21.191210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:21.192608 (+ 1398us) server_negotiation.cc:407] Connection header received
0619 06:51:21.194755 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:21.194761 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:21.194858 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:21.194977 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:21.197688 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.198746 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.201311 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.201973 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.202103 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.207188 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:21.207197 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:21.207206 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:21.207264 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:21.211510 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.212596 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.212600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.212602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.212710 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.213368 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.213373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.213375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.213763 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:21.213964 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:21.214985 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0619 06:51:21.215206 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:51:21.216596 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.191667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.66:39915)
0619 06:51:21.192454 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:21.192500 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:21.194474 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:21.195309 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:21.195322 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:21.196284 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:21.197444 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.197472 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.198912 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.198920 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.201150 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.201162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.202239 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.202246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.202562 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.203445 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:21.203486 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:21.206892 (+ 3406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:21.211713 (+ 4821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.211727 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.211752 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.212422 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.212836 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.212845 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.212851 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.213198 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.214032 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:21.214042 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:21.214575 (+ 533us) client_negotiation.cc:769] Sending connection context
0619 06:51:21.214893 (+ 318us) client_negotiation.cc:240] Negotiation successful
0619 06:51:21.215436 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":341,"threads_started":1}
I20260619 06:51:21.240013 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.224479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:43267 (local address 127.18.39.126:42837)
0619 06:51:21.224732 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:21.224741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:21.225401 (+ 660us) server_negotiation.cc:407] Connection header received
0619 06:51:21.225629 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:21.225632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:21.225690 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:21.225767 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:21.227235 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.227938 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.230461 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.231131 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.231268 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.233366 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:21.233375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:21.233382 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:21.233439 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:21.236641 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.237591 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.237598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.237603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.237759 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.238242 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.238251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.238256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.238822 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:21.239004 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:21.239447 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 06:51:21.239665 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:51:21.240123 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.224474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.66:43267)
0619 06:51:21.225238 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:21.225270 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:21.225493 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:21.225827 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:21.225835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:21.226595 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:21.227061 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.227077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.228080 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.228088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.230277 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.230300 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.231325 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.231334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.231518 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.232236 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:21.232253 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:21.233164 (+ 911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:21.236774 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.236779 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.236783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.237414 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.237886 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.237891 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.237895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.238093 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.238952 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:21.238959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:21.239195 (+ 236us) client_negotiation.cc:769] Sending connection context
0619 06:51:21.239350 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:51:21.239681 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":394,"threads_started":1}
I20260619 06:51:21.246146 26398 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:43041
I20260619 06:51:21.246174 26532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:43041 every 8 connection(s)
I20260619 06:51:21.247334 26398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:21.249413 26533 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42837
I20260619 06:51:21.249655 26533 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:21.250277 26533 heartbeater.cc:511] Master 127.18.39.126:42837 requested a full tablet report, sending...
I20260619 06:51:21.251598 26187 ts_manager.cc:194] Registered new tserver with Master: 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:21.252408 26187 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:39915
I20260619 06:51:21.255548 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26398
I20260619 06:51:21.255710 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:21.256160 18589 external_mini_cluster.cc:1428] Setting key bd46daf5e9200911c58394be018492c4
WARNING: no policy specified for oryx/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
I20260619 06:51:21.302676 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42837
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:21.632030 26537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:21.632320 26537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:21.632474 26537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:21.640192 26537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:21.640313 26537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:21.640539 26537 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:51:21.640666 26537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:21.650503 26537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.39.126:42837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:21.651829 26537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:21.653159 26537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:21.666391 26542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:21.666424 26545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:21.666391 26543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:21.667395 26537 server_base.cc:1034] running on GCE node
I20260619 06:51:21.668048 26537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:21.669003 26537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:21.670223 26537 hybrid_clock.cc:648] HybridClock initialized: now 1781851881670132 us; error 58 us; skew 500 ppm
I20260619 06:51:21.670529 26537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:21.673646 26537 init.cc:377] Logged in from keytab as oryx/127.18.39.67@KRBTEST.COM (short username oryx)
I20260619 06:51:21.675033 26537 webserver.cc:460] Webserver started at http://127.18.39.67:42183/ using document root <none> and password file <none>
I20260619 06:51:21.675750 26537 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:21.675889 26537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:21.676226 26537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:21.679206 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "210e69b32f6a4b428c46fd0dad850087"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "34e40bf6ccbf6e5bfe0c69f590d2234f"
server_key_iv: "99927a942cf1a2f9474a268693470b28"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.679996 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "210e69b32f6a4b428c46fd0dad850087"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "34e40bf6ccbf6e5bfe0c69f590d2234f"
server_key_iv: "99927a942cf1a2f9474a268693470b28"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.685173 26537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:51:21.688768 26552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:21.690261 26537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:21.690439 26537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "210e69b32f6a4b428c46fd0dad850087"
format_stamp: "Formatted at 2026-06-19 06:51:21 on dist-test-slave-1h5s"
server_key: "34e40bf6ccbf6e5bfe0c69f590d2234f"
server_key_iv: "99927a942cf1a2f9474a268693470b28"
server_key_version: "encryptionkey@0"
I20260619 06:51:21.690667 26537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:21.708950 26537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:21.713197 26537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:21.713507 26537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:21.716377 26537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:21.716485 26537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:21.716672 26537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:21.716747 26537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:21.727309 26394 heartbeater.cc:503] Master 127.18.39.126:42837 was elected leader, sending a full tablet report...
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:21.744614 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.717637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34021 (local address 127.18.39.126:42837)
0619 06:51:21.717894 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:21.717900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:21.719020 (+ 1120us) server_negotiation.cc:407] Connection header received
0619 06:51:21.722238 (+ 3218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:21.722246 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:21.722356 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:21.722458 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:21.725603 (+ 3145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.727044 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.730404 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.731184 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.731358 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.737121 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:21.737129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:21.737138 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:21.737199 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:21.740781 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.741993 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.741998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.742000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.742129 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.742778 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.742783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.742787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.743084 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:21.743250 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:21.744014 (+ 764us) server_negotiation.cc:299] Negotiation successful
0619 06:51:21.744253 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 06:51:21.745855 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.717855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.67:34021)
0619 06:51:21.718827 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:21.718887 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:21.721840 (+ 2953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:21.723112 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:21.723125 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:21.724217 (+ 1092us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:21.725304 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.725356 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.727327 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.727335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.730234 (+ 2899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.730255 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.731489 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.731498 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.731747 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.732760 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:21.732803 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:21.736794 (+ 3991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:21.740951 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.740967 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.740994 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.741841 (+ 847us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.742249 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.742257 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.742263 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.742620 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.743227 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:21.743236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:21.743632 (+ 396us) client_negotiation.cc:769] Sending connection context
0619 06:51:21.743900 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 06:51:21.744452 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":390,"threads_started":1}
I20260619 06:51:21.769465 26537 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:43023
I20260619 06:51:21.769485 26671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:43023 every 8 connection(s)
I20260619 06:51:21.770903 26537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:21.773190 26672 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42837
I20260619 06:51:21.773474 26672 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:21.774201 26672 heartbeater.cc:511] Master 127.18.39.126:42837 requested a full tablet report, sending...
I20260619 06:51:21.775970 26187 ts_manager.cc:194] Registered new tserver with Master: 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023)
I20260619 06:51:21.776324 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26537
I20260619 06:51:21.776490 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:21.777022 18589 external_mini_cluster.cc:1428] Setting key 1ece21dce6954471d42643dfbaf80965
I20260619 06:51:21.777173 26187 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:34021
I20260619 06:51:21.780058 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.765942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:54005 (local address 127.18.39.126:42837)
0619 06:51:21.766117 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:21.766122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:21.766650 (+ 528us) server_negotiation.cc:407] Connection header received
0619 06:51:21.766972 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:21.766978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:21.767061 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:21.767186 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:21.768264 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.769165 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.770573 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.771004 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.771146 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.773681 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:21.773686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:21.773689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:21.773722 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:21.776957 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.777887 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.777892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.777896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.778050 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:21.778461 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:21.778466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:21.778469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:21.778794 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:21.778933 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:21.779449 (+ 516us) server_negotiation.cc:299] Negotiation successful
0619 06:51:21.779666 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:51:21.780071 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:21.765895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.18.39.67:54005)
0619 06:51:21.766499 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:21.766532 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:21.766735 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:21.767221 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:21.767225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:21.767757 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:21.768098 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.768109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.769314 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.769319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.770414 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:21.770427 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.771238 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.771246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:21.771498 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:21.772342 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:21.772368 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:21.773431 (+ 1063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:21.777144 (+ 3713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.777154 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.777156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.777747 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.778163 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:21.778168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:21.778171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:21.778318 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:21.778993 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:21.778999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:21.779179 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 06:51:21.779317 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 06:51:21.779629 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":325,"threads_started":1}
I20260619 06:51:21.780208 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:21 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:22.254910 26533 heartbeater.cc:503] Master 127.18.39.126:42837 was elected leader, sending a full tablet report...
I20260619 06:51:22.780705 26672 heartbeater.cc:503] Master 127.18.39.126:42837 was elected leader, sending a full tablet report...
W20260619 06:51:22.887705 26680 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.062s user 0.001s sys 0.014s
W20260619 06:51:22.888908 26281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:21.819901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.18.39.126:42837)
0619 06:51:21.820154 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:21.820166 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:21.820440 (+ 274us) server_negotiation.cc:407] Connection header received
0619 06:51:21.820671 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:21.820675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:21.820740 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:21.820844 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:21.822278 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.823010 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.824256 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:21.824674 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:21.824811 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.888557 (+1063746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56342: BlockingRecv error: recv got EOF from 127.0.0.1:56342 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 06:51:22 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:22.911554 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.895169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.18.39.126:42837)
0619 06:51:22.895329 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:22.895333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:22.895653 (+ 320us) server_negotiation.cc:407] Connection header received
0619 06:51:22.895842 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:22.895846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:22.895945 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:22.896039 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:22.897271 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.898074 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.899332 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.899731 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.899864 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.906601 (+ 6737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:22.906607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:22.906610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:22.906648 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:22.909134 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:22.909834 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:22.909838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:22.909840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:22.909968 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:22.910338 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:22.910342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:22.910344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:22.910635 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:22.910873 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:22.911085 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 06:51:22.911272 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 06:51:22.915716 26187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:22.922859 26187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:22.947770 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.937781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.18.39.67:43023)
0619 06:51:22.940398 (+ 2617us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:22.940407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:22.940434 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:51:22.940522 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:22.940530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:22.940745 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:22.940946 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:22.941714 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.942757 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.945707 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.946659 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.946799 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.946993 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:22.947264 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 06:51:22.947419 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":2292,"threads_started":1}
I20260619 06:51:22.947825 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.937240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43023 (local address 127.0.0.1:32774)
0619 06:51:22.939149 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:22.939180 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:22.939373 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:22.940971 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:22.940976 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:22.941000 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:22.941519 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.941539 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.942890 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.942895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.945519 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.945534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.946851 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.946857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.947040 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.947087 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 06:51:22.947216 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 06:51:22.947406 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1724,"mutex_wait_us":86,"thread_start_us":1525,"threads_started":1}
I20260619 06:51:22.948297 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.935905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.18.39.66:43041)
0619 06:51:22.937087 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:22.937097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:22.937728 (+ 631us) server_negotiation.cc:407] Connection header received
0619 06:51:22.938040 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:22.938051 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:22.938336 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:22.938505 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:22.939673 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.941999 (+ 2326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.944636 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.946618 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.946741 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.946973 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:22.947773 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 06:51:22.947962 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":917,"threads_started":1}
I20260619 06:51:22.948408 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.935371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:45183 (local address 127.0.0.1:39732)
0619 06:51:22.936292 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:22.936333 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:22.936494 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:22.940600 (+ 4106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:22.940605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:22.940629 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:22.941037 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.941054 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.943042 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.943050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.945534 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.945543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.947494 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.947503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.947761 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.947798 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:51:22.947868 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 06:51:22.947992 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":48,"thread_start_us":681,"threads_started":1}
I20260619 06:51:22.949255 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.935614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43041 (local address 127.0.0.1:42574)
0619 06:51:22.937604 (+ 1990us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:22.937627 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:22.937840 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:22.938848 (+ 1008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:22.938856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:22.938882 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:22.939418 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.939434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.942152 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.942158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.944238 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:22.944254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.947177 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.947184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:22.947399 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.947449 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 06:51:22.947540 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 06:51:22.948557 (+ 1017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1841,"mutex_wait_us":34,"thread_start_us":1655,"threads_started":1}
I20260619 06:51:22.949559 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:22.935596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39732 (local address 127.18.39.65:45183)
0619 06:51:22.940129 (+ 4533us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:22.940137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:22.940163 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:51:22.940248 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:22.940254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:22.940434 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:22.940620 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:22.941210 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.942843 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.945681 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:22.947235 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:22.948619 (+ 1384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:22.948889 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:22.948961 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 06:51:22.949125 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4343,"thread_start_us":4173,"threads_started":1}
I20260619 06:51:22.952083 26324 tablet_service.cc:1467] Processing CreateTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 (DEFAULT_TABLE table=test-table [id=45b063fb14374a50b178cc11a177b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:22.952121 26602 tablet_service.cc:1467] Processing CreateTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 (DEFAULT_TABLE table=test-table [id=45b063fb14374a50b178cc11a177b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:22.952538 26463 tablet_service.cc:1467] Processing CreateTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 (DEFAULT_TABLE table=test-table [id=45b063fb14374a50b178cc11a177b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:22.953188 26324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5c2d2bd37453bbb670a595512d0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:22.953441 26602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5c2d2bd37453bbb670a595512d0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:22.953683 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc5c2d2bd37453bbb670a595512d0f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:22.963950 26697 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Bootstrap starting.
I20260619 06:51:22.967482 26697 tablet_bootstrap.cc:654] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:22.967514 26698 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Bootstrap starting.
I20260619 06:51:22.968981 26697 log.cc:824] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:22.970996 26698 tablet_bootstrap.cc:654] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:22.972254 26698 log.cc:824] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:22.972332 26697 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: No bootstrap required, opened a new log
I20260619 06:51:22.972592 26697 ts_tablet_manager.cc:1397] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:51:22.973959 26700 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Bootstrap starting.
I20260619 06:51:22.975296 26698 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: No bootstrap required, opened a new log
I20260619 06:51:22.975591 26698 ts_tablet_manager.cc:1397] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:51:22.977555 26700 tablet_bootstrap.cc:654] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:22.978799 26700 log.cc:824] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:22.981891 26700 tablet_bootstrap.cc:492] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: No bootstrap required, opened a new log
I20260619 06:51:22.982162 26700 ts_tablet_manager.cc:1397] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260619 06:51:22.982861 26697 raft_consensus.cc:348] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.983314 26697 raft_consensus.cc:374] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:22.983430 26697 raft_consensus.cc:729] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Initialized, Role: FOLLOWER
I20260619 06:51:22.983428 26698 raft_consensus.cc:348] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.983862 26698 raft_consensus.cc:374] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:22.983980 26698 raft_consensus.cc:729] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Initialized, Role: FOLLOWER
I20260619 06:51:22.984068 26697 consensus_queue.cc:260] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.984612 26698 consensus_queue.cc:260] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.985803 26697 ts_tablet_manager.cc:1428] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260619 06:51:22.987264 26698 ts_tablet_manager.cc:1428] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260619 06:51:22.992142 26700 raft_consensus.cc:348] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.992509 26700 raft_consensus.cc:374] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:22.992606 26700 raft_consensus.cc:729] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Initialized, Role: FOLLOWER
I20260619 06:51:22.993151 26700 consensus_queue.cc:260] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:22.994736 26700 ts_tablet_manager.cc:1428] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260619 06:51:22.999899 26534 tablet.cc:2369] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:23.023499 26673 tablet.cc:2369] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:23.220399 26395 tablet.cc:2369] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:51:23.272095 26703 raft_consensus.cc:482] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:23.272449 26703 raft_consensus.cc:504] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.274348 26703 leader_election.cc:288] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041), 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023)
I20260619 06:51:23.281205 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.274925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:34579 (local address 127.18.39.66:43041)
0619 06:51:23.275107 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.275114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.275837 (+ 723us) server_negotiation.cc:407] Connection header received
0619 06:51:23.276095 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.276100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.276170 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.276266 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:23.276822 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.277623 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.279520 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.280277 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.280471 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.280548 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.280819 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.280946 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:51:23.281286 26696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.274867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42441 (local address 127.18.39.67:43023)
0619 06:51:23.275097 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.275104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.276261 (+ 1157us) server_negotiation.cc:407] Connection header received
0619 06:51:23.276459 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.276463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.276522 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.276619 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:23.277182 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.277927 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.279636 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.280374 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.280556 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.280641 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.280915 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.281040 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:51:23.281407 26707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.274990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43023 (local address 127.18.39.65:42441)
0619 06:51:23.276149 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.276164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.276304 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.276638 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.276642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.276674 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:23.277004 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.277030 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.278065 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.278072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.279483 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.279493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.280599 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.280605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.280753 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.280775 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.280878 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.281007 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"mutex_wait_us":30,"thread_start_us":808,"threads_started":1}
I20260619 06:51:23.281828 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.274990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43041 (local address 127.18.39.65:34579)
0619 06:51:23.275707 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.275726 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.275926 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.276343 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.276347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.276384 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:23.276682 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.276690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.277742 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.277749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.279356 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.279368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.280491 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.280497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.280664 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.280689 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.280794 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.280930 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"mutex_wait_us":58,"thread_start_us":405,"threads_started":1}
I20260619 06:51:23.282543 26624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5c2d2bd37453bbb670a595512d0f3" candidate_uuid: "bb494569ce15479ab2d304c1e0d03028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210e69b32f6a4b428c46fd0dad850087" is_pre_election: true
I20260619 06:51:23.282801 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5c2d2bd37453bbb670a595512d0f3" candidate_uuid: "bb494569ce15479ab2d304c1e0d03028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495" is_pre_election: true
I20260619 06:51:23.283058 26624 raft_consensus.cc:2454] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb494569ce15479ab2d304c1e0d03028 in term 0.
I20260619 06:51:23.283315 26483 raft_consensus.cc:2454] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb494569ce15479ab2d304c1e0d03028 in term 0.
I20260619 06:51:23.283803 26276 leader_election.cc:302] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:23.284262 26703 raft_consensus.cc:2781] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:23.284413 26703 raft_consensus.cc:482] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:23.284533 26703 raft_consensus.cc:3037] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.286837 26703 raft_consensus.cc:504] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.287552 26703 leader_election.cc:288] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [CANDIDATE]: Term 1 election: Requested vote from peers 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041), 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023)
I20260619 06:51:23.288086 26624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5c2d2bd37453bbb670a595512d0f3" candidate_uuid: "bb494569ce15479ab2d304c1e0d03028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210e69b32f6a4b428c46fd0dad850087"
I20260619 06:51:23.288085 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc5c2d2bd37453bbb670a595512d0f3" candidate_uuid: "bb494569ce15479ab2d304c1e0d03028" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495"
I20260619 06:51:23.288295 26624 raft_consensus.cc:3037] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.288352 26483 raft_consensus.cc:3037] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.290838 26624 raft_consensus.cc:2454] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb494569ce15479ab2d304c1e0d03028 in term 1.
I20260619 06:51:23.290959 26483 raft_consensus.cc:2454] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb494569ce15479ab2d304c1e0d03028 in term 1.
I20260619 06:51:23.291388 26276 leader_election.cc:302] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:23.291746 26703 raft_consensus.cc:2781] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:23.292208 26703 raft_consensus.cc:686] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 1 LEADER]: Becoming Leader. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Running, Role: LEADER
I20260619 06:51:23.292737 26703 consensus_queue.cc:237] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.297737 26185 catalog_manager.cc:5495] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb494569ce15479ab2d304c1e0d03028 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "bb494569ce15479ab2d304c1e0d03028" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } health_report { overall_health: UNKNOWN } } }
I20260619 06:51:23.313819 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.308957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.18.39.65:45183)
0619 06:51:23.309126 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.309130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.309243 (+ 113us) server_negotiation.cc:407] Connection header received
0619 06:51:23.309491 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.309495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.309573 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.309709 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:23.309993 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.310707 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.311918 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.312314 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.312413 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.312769 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.313294 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.313505 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.313548 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.313613 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:51:23.321978 26483 raft_consensus.cc:1261] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Refusing update from remote peer bb494569ce15479ab2d304c1e0d03028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.321980 26624 raft_consensus.cc:1261] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Refusing update from remote peer bb494569ce15479ab2d304c1e0d03028: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.322939 26708 consensus_queue.cc:1035] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:23.323292 26703 consensus_queue.cc:1035] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:23.351969 26186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:23.352259 26186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:23.356323 26186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Sending DeleteTablet for 3 replicas of tablet afc5c2d2bd37453bbb670a595512d0f3
I20260619 06:51:23.357614 26324 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:23 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:39732
I20260619 06:51:23.357878 26463 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:23 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42574
I20260619 06:51:23.358485 26602 tablet_service.cc:1514] Processing DeleteTablet for tablet afc5c2d2bd37453bbb670a595512d0f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:23 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:32774
I20260619 06:51:23.359889 26185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:23.360630 26185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:23.370790 26721 tablet_replica.cc:330] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: stopping tablet replica
I20260619 06:51:23.371637 26721 raft_consensus.cc:2229] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:23.372473 26323 tablet_service.cc:1467] Processing CreateTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 (DEFAULT_TABLE table=test-table [id=85803c1821c24758b0ecedd8631e135e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:23.372535 26601 tablet_service.cc:1467] Processing CreateTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 (DEFAULT_TABLE table=test-table [id=85803c1821c24758b0ecedd8631e135e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:23.373209 26601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89f2ded18f34fe88e5d860b31c377b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.373240 26323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89f2ded18f34fe88e5d860b31c377b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.381867 26463 tablet_service.cc:1467] Processing CreateTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 (DEFAULT_TABLE table=test-table [id=85803c1821c24758b0ecedd8631e135e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:23.382592 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d89f2ded18f34fe88e5d860b31c377b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.386096 26697 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Bootstrap starting.
I20260619 06:51:23.388451 26697 tablet_bootstrap.cc:654] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.393239 26722 tablet_replica.cc:330] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: stopping tablet replica
I20260619 06:51:23.393750 26722 raft_consensus.cc:2229] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:23.394336 26722 raft_consensus.cc:2258] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:23.394707 26721 raft_consensus.cc:2258] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:23.396570 26720 tablet_replica.cc:330] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: stopping tablet replica
I20260619 06:51:23.396688 26698 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Bootstrap starting.
I20260619 06:51:23.397001 26720 raft_consensus.cc:2229] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:23.397666 26720 raft_consensus.cc:2258] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:23.398046 26722 ts_tablet_manager.cc:1905] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:23.398342 26698 tablet_bootstrap.cc:654] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.400604 26698 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: No bootstrap required, opened a new log
I20260619 06:51:23.400743 26698 ts_tablet_manager.cc:1397] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 06:51:23.401528 26698 raft_consensus.cc:348] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.401749 26698 raft_consensus.cc:374] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.401855 26698 raft_consensus.cc:729] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.402238 26720 ts_tablet_manager.cc:1905] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:23.402144 26698 consensus_queue.cc:260] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.403667 26698 ts_tablet_manager.cc:1428] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:23.405498 26721 ts_tablet_manager.cc:1905] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:23.405772 26722 ts_tablet_manager.cc:1918] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:23.405936 26722 log.cc:1196] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/afc5c2d2bd37453bbb670a595512d0f3
I20260619 06:51:23.406498 26722 ts_tablet_manager.cc:1939] T afc5c2d2bd37453bbb670a595512d0f3 P 210e69b32f6a4b428c46fd0dad850087: Deleting consensus metadata
I20260619 06:51:23.408252 26173 catalog_manager.cc:4841] TS 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023): tablet afc5c2d2bd37453bbb670a595512d0f3 (table test-table [id=45b063fb14374a50b178cc11a177b79c]) successfully deleted
I20260619 06:51:23.414867 26721 ts_tablet_manager.cc:1918] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:23.415143 26721 log.cc:1196] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/afc5c2d2bd37453bbb670a595512d0f3
I20260619 06:51:23.415529 26697 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: No bootstrap required, opened a new log
I20260619 06:51:23.415702 26697 ts_tablet_manager.cc:1397] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Time spent bootstrapping tablet: real 0.030s user 0.009s sys 0.000s
I20260619 06:51:23.415855 26721 ts_tablet_manager.cc:1939] T afc5c2d2bd37453bbb670a595512d0f3 P 7e700c4a588d43139c79516c4622e495: Deleting consensus metadata
I20260619 06:51:23.416121 26720 ts_tablet_manager.cc:1918] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:23.416278 26720 log.cc:1196] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/afc5c2d2bd37453bbb670a595512d0f3
I20260619 06:51:23.416672 26697 raft_consensus.cc:348] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.416849 26720 ts_tablet_manager.cc:1939] T afc5c2d2bd37453bbb670a595512d0f3 P bb494569ce15479ab2d304c1e0d03028: Deleting consensus metadata
I20260619 06:51:23.416911 26697 raft_consensus.cc:374] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.417023 26697 raft_consensus.cc:729] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.417320 26697 consensus_queue.cc:260] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.418109 26700 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Bootstrap starting.
I20260619 06:51:23.418157 26172 catalog_manager.cc:4841] TS 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041): tablet afc5c2d2bd37453bbb670a595512d0f3 (table test-table [id=45b063fb14374a50b178cc11a177b79c]) successfully deleted
I20260619 06:51:23.418620 26697 ts_tablet_manager.cc:1428] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 06:51:23.419423 26173 catalog_manager.cc:4841] TS bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183): tablet afc5c2d2bd37453bbb670a595512d0f3 (table test-table [id=45b063fb14374a50b178cc11a177b79c]) successfully deleted
I20260619 06:51:23.419665 26700 tablet_bootstrap.cc:654] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.421581 26700 tablet_bootstrap.cc:492] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: No bootstrap required, opened a new log
I20260619 06:51:23.421772 26700 ts_tablet_manager.cc:1397] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260619 06:51:23.422375 26700 raft_consensus.cc:348] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.422557 26700 raft_consensus.cc:374] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.422660 26700 raft_consensus.cc:729] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.422927 26700 consensus_queue.cc:260] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.423434 26700 ts_tablet_manager.cc:1428] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 06:51:23.600109 26704 raft_consensus.cc:482] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:23.600400 26704 raft_consensus.cc:504] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.602078 26704 leader_election.cc:288] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:23.610083 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.603053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60371 (local address 127.18.39.65:45183)
0619 06:51:23.603242 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.603247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.603266 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:23.603340 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.603345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.603454 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.603552 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:23.604379 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.605476 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.607648 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.608961 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.609133 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.609207 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.609605 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.609729 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:51:23.610116 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.602310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:45183 (local address 127.18.39.67:60371)
0619 06:51:23.602929 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.602952 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.603101 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.603780 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.603785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.603820 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:23.604213 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.604227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.605609 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.605614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.607464 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.607478 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.609187 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.609195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.609426 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.609477 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.609580 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.609715 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":5,"thread_start_us":304,"threads_started":1}
I20260619 06:51:23.610242 26694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.603400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:58391 (local address 127.18.39.66:43041)
0619 06:51:23.603596 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.603603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.604019 (+ 416us) server_negotiation.cc:407] Connection header received
0619 06:51:23.604355 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.604361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.604444 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.604555 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:23.605175 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.606297 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.608304 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.609148 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.609309 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.609400 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.609811 (+ 411us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.609960 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 06:51:23.610877 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.602608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43041 (local address 127.18.39.67:58391)
0619 06:51:23.603913 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.603942 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.604131 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.604593 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.604597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.604627 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:23.604992 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.605004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.606440 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.606446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.608146 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.608156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.609418 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.609424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.609613 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.609650 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.610390 (+ 740us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.610531 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"thread_start_us":1001,"threads_started":1}
I20260619 06:51:23.611549 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89f2ded18f34fe88e5d860b31c377b0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495" is_pre_election: true
I20260619 06:51:23.611562 26344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89f2ded18f34fe88e5d860b31c377b0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028" is_pre_election: true
I20260619 06:51:23.611809 26483 raft_consensus.cc:2454] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:23.612067 26344 raft_consensus.cc:2454] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:23.612291 26554 leader_election.cc:302] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, 7e700c4a588d43139c79516c4622e495; no voters:
I20260619 06:51:23.612689 26704 raft_consensus.cc:2781] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:23.612833 26704 raft_consensus.cc:482] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:23.612931 26704 raft_consensus.cc:3037] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.615060 26704 raft_consensus.cc:504] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.615705 26704 leader_election.cc:288] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Requested vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:23.616158 26344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89f2ded18f34fe88e5d860b31c377b0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028"
I20260619 06:51:23.616243 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d89f2ded18f34fe88e5d860b31c377b0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495"
I20260619 06:51:23.616417 26344 raft_consensus.cc:3037] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.616469 26483 raft_consensus.cc:3037] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.619040 26344 raft_consensus.cc:2454] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:23.619496 26555 leader_election.cc:302] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:23.619565 26483 raft_consensus.cc:2454] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:23.619799 26704 raft_consensus.cc:2781] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:23.620338 26704 raft_consensus.cc:686] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 LEADER]: Becoming Leader. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Running, Role: LEADER
I20260619 06:51:23.620735 26704 consensus_queue.cc:237] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.624490 26186 catalog_manager.cc:5495] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 reported cstate change: term changed from 0 to 1, leader changed from <none> to 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } health_report { overall_health: HEALTHY } } }
Jun 19 06:51:23 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851880, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.126@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:23.718206 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.703757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42837 (local address 127.0.0.1:56354)
0619 06:51:23.704480 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.704517 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.704703 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.705049 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.705053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.705788 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:23.705968 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.705977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.706991 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.706998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.707942 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.707953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.708650 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.708656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.708836 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.709494 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:23.709518 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:23.711954 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:23.715106 (+ 3152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:23.715120 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:23.715129 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:23.715746 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:23.716142 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:23.716149 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:23.716153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:23.716333 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:23.716938 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:23.716947 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:23.717146 (+ 199us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.717387 (+ 241us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.717706 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":400,"threads_started":1}
I20260619 06:51:23.718374 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.703898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.18.39.126:42837)
0619 06:51:23.704771 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.704779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.704803 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:51:23.704854 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.704857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.704914 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.705029 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:23.706110 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.706857 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.708075 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.708494 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.708631 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.712151 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:23.712156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:23.712159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:23.712187 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:23.714953 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:23.715862 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:23.715868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:23.715872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:23.716017 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:23.716447 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:23.716452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:23.716456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:23.716786 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:23.716920 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.717488 (+ 568us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.717864 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":757,"thread_start_us":560,"threads_started":1}
I20260619 06:51:23.724761 26186 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:56354:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:51:23.726100 26186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:23.733973 26324 tablet_service.cc:1467] Processing CreateTablet for tablet 61def0813410419fa093c5be582095f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d678694b58c4428c98d04029bdad6709]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:51:23.734709 26324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61def0813410419fa093c5be582095f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.736544 26601 tablet_service.cc:1467] Processing CreateTablet for tablet 61def0813410419fa093c5be582095f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d678694b58c4428c98d04029bdad6709]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:51:23.737048 26601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61def0813410419fa093c5be582095f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.743558 26697 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: Bootstrap starting.
I20260619 06:51:23.744441 26698 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: Bootstrap starting.
I20260619 06:51:23.745007 26463 tablet_service.cc:1467] Processing CreateTablet for tablet 61def0813410419fa093c5be582095f4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d678694b58c4428c98d04029bdad6709]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:51:23.745417 26697 tablet_bootstrap.cc:654] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.745575 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61def0813410419fa093c5be582095f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:23.746217 26698 tablet_bootstrap.cc:654] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.754058 26700 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: Bootstrap starting.
I20260619 06:51:23.755787 26700 tablet_bootstrap.cc:654] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:23.756316 26697 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: No bootstrap required, opened a new log
I20260619 06:51:23.756455 26697 ts_tablet_manager.cc:1397] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260619 06:51:23.757311 26697 raft_consensus.cc:348] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.757601 26697 raft_consensus.cc:374] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.757686 26697 raft_consensus.cc:729] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.757975 26700 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: No bootstrap required, opened a new log
I20260619 06:51:23.757930 26697 consensus_queue.cc:260] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.758129 26700 ts_tablet_manager.cc:1397] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260619 06:51:23.758471 26703 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.758724 26703 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.758761 26700 raft_consensus.cc:348] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.758966 26700 raft_consensus.cc:374] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.759032 26700 raft_consensus.cc:729] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.759239 26700 consensus_queue.cc:260] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.759696 26697 ts_tablet_manager.cc:1428] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 06:51:23.759634 26705 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.759902 26705 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.760716 26700 ts_tablet_manager.cc:1428] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260619 06:51:23.761590 26698 tablet_bootstrap.cc:492] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: No bootstrap required, opened a new log
I20260619 06:51:23.761745 26698 ts_tablet_manager.cc:1397] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.008s
I20260619 06:51:23.763074 26698 raft_consensus.cc:348] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.763326 26698 raft_consensus.cc:374] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:23.763391 26698 raft_consensus.cc:729] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Initialized, Role: FOLLOWER
I20260619 06:51:23.763656 26698 consensus_queue.cc:260] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.764042 26704 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.764315 26704 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.764950 26698 ts_tablet_manager.cc:1428] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:23.859071 26704 raft_consensus.cc:482] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:23.859349 26704 raft_consensus.cc:504] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.859947 26704 leader_election.cc:288] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:23.860553 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61def0813410419fa093c5be582095f4" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495" is_pre_election: true
I20260619 06:51:23.860586 26344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61def0813410419fa093c5be582095f4" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028" is_pre_election: true
I20260619 06:51:23.860821 26483 raft_consensus.cc:2454] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:23.860857 26344 raft_consensus.cc:2454] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:23.861270 26554 leader_election.cc:302] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, 7e700c4a588d43139c79516c4622e495, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:23.861655 26704 raft_consensus.cc:2781] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:23.861744 26704 raft_consensus.cc:482] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:23.861864 26704 raft_consensus.cc:3037] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.864078 26704 raft_consensus.cc:504] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.864753 26704 leader_election.cc:288] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Requested vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:23.865116 26344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61def0813410419fa093c5be582095f4" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028"
I20260619 06:51:23.865254 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61def0813410419fa093c5be582095f4" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495"
I20260619 06:51:23.865329 26344 raft_consensus.cc:3037] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.865520 26483 raft_consensus.cc:3037] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:23.867383 26344 raft_consensus.cc:2454] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:23.867640 26483 raft_consensus.cc:2454] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:23.867786 26555 leader_election.cc:302] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:23.868070 26704 raft_consensus.cc:2781] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:23.868251 26704 raft_consensus.cc:686] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [term 1 LEADER]: Becoming Leader. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Running, Role: LEADER
I20260619 06:51:23.868541 26704 consensus_queue.cc:237] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:23.869915 26728 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: TxnStatusTablet state changed. Reason: New leader 210e69b32f6a4b428c46fd0dad850087. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.870157 26728 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:51:23.870955 26741 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:51:23.871232 26741 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:51:23.871469 26185 catalog_manager.cc:5495] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 reported cstate change: term changed from 0 to 1, leader changed from <none> to 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } health_report { overall_health: HEALTHY } } }
I20260619 06:51:23.909148 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.902163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32780 (local address 127.18.39.67:43023)
0619 06:51:23.902845 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.902850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.902870 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:51:23.902949 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.902952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.903027 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.903153 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:23.903627 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.904686 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.906615 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.907045 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.907143 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.907747 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.908388 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.908502 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.908662 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.908762 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":421,"threads_started":1}
I20260619 06:51:23.909192 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.902160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43023 (local address 127.0.0.1:32780)
0619 06:51:23.902415 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.902436 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.902595 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.903203 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.903208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.903221 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:51:23.903417 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.903429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.904816 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.904822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.906460 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.906476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.907201 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.907207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.907425 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.907503 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.908496 (+ 993us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.908519 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.908624 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.908829 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 06:51:23.913774 26483 raft_consensus.cc:1261] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.913872 26344 raft_consensus.cc:1261] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.914551 26728 consensus_queue.cc:1035] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:23.914875 26704 consensus_queue.cc:1035] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:23.923892 26703 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: TxnStatusTablet state changed. Reason: New leader 210e69b32f6a4b428c46fd0dad850087. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.924191 26703 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.924865 26705 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: TxnStatusTablet state changed. Reason: New leader 210e69b32f6a4b428c46fd0dad850087. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.925168 26705 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.925854 26745 mvcc.cc:205] Tried to move back new op lower bound from 7298465316503584768 to 7298465316327403520. Current Snapshot: MvccSnapshot[applied={T|T < 7298465316503584768}]
I20260619 06:51:23.926702 26728 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.926993 26728 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:51:23.928073 26728 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.928431 26728 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 210e69b32f6a4b428c46fd0dad850087: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:51:23.931258 26746 mvcc.cc:205] Tried to move back new op lower bound from 7298465316503584768 to 7298465316327403520. Current Snapshot: MvccSnapshot[applied={T|T < 7298465316503584768}]
I20260619 06:51:23.948612 26703 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.948988 26703 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P bb494569ce15479ab2d304c1e0d03028: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.953999 26705 tablet_replica.cc:439] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } } }
I20260619 06:51:23.954267 26705 tablet_replica.cc:442] T 61def0813410419fa093c5be582095f4 P 7e700c4a588d43139c79516c4622e495: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:51:23.959419 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.946444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32784 (local address 127.18.39.67:43023)
0619 06:51:23.946696 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.946706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.953161 (+ 6455us) server_negotiation.cc:407] Connection header received
0619 06:51:23.953430 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.953436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.953547 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.953646 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:23.954521 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.955607 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.957626 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.958074 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.958169 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.958703 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.958823 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.958903 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.959051 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.959112 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 06:51:23.977214 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.971139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:39039 (local address 127.18.39.67:43023)
0619 06:51:23.971388 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:23.971395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:23.972150 (+ 755us) server_negotiation.cc:407] Connection header received
0619 06:51:23.972348 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:23.972353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:23.972447 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:23.972539 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:23.973030 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.973853 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.975329 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.975901 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.976035 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.976456 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.976555 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.976647 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:23.976779 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 06:51:23.976847 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:51:23.977358 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:23.971156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43023 (local address 127.18.39.67:39039)
0619 06:51:23.972050 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:23.972066 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:23.972233 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:23.972559 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:23.972562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:23.972573 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:51:23.972894 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.972903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.973965 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.973973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.975191 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:23.975203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:23.976086 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:23.976095 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:23.976225 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:23.976286 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:51:23.976648 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:51:23.976660 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:51:23.976754 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:51:23.976893 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":552,"threads_started":1}
I20260619 06:51:23.986240 26483 raft_consensus.cc:1261] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.986428 26344 raft_consensus.cc:1261] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:23.986783 26751 consensus_queue.cc:1035] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:23.987048 26704 consensus_queue.cc:1035] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:24.051630 26186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.051832 26186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.054692 26186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Sending DeleteTablet for 3 replicas of tablet d89f2ded18f34fe88e5d860b31c377b0
I20260619 06:51:24.055564 26602 tablet_service.cc:1514] Processing DeleteTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:32774
I20260619 06:51:24.055593 26324 tablet_service.cc:1514] Processing DeleteTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:39732
I20260619 06:51:24.056221 26463 tablet_service.cc:1514] Processing DeleteTablet for tablet d89f2ded18f34fe88e5d860b31c377b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42574
I20260619 06:51:24.064733 26764 tablet_replica.cc:330] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: stopping tablet replica
I20260619 06:51:24.065008 26764 raft_consensus.cc:2229] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.065292 26764 raft_consensus.cc:2258] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.067610 26764 ts_tablet_manager.cc:1905] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.070685 26763 tablet_replica.cc:330] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: stopping tablet replica
I20260619 06:51:24.070899 26763 raft_consensus.cc:2229] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:24.071535 26763 raft_consensus.cc:2258] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.072646 26763 ts_tablet_manager.cc:1905] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.073009 26762 tablet_replica.cc:330] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: stopping tablet replica
I20260619 06:51:24.073243 26762 raft_consensus.cc:2229] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.073534 26762 raft_consensus.cc:2258] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.074574 26762 ts_tablet_manager.cc:1905] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.075613 26764 ts_tablet_manager.cc:1918] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.075748 26764 log.cc:1196] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/d89f2ded18f34fe88e5d860b31c377b0
I20260619 06:51:24.076201 26764 ts_tablet_manager.cc:1939] T d89f2ded18f34fe88e5d860b31c377b0 P 7e700c4a588d43139c79516c4622e495: Deleting consensus metadata
I20260619 06:51:24.077419 26172 catalog_manager.cc:4841] TS 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041): tablet d89f2ded18f34fe88e5d860b31c377b0 (table test-table [id=85803c1821c24758b0ecedd8631e135e]) successfully deleted
I20260619 06:51:24.080888 26763 ts_tablet_manager.cc:1918] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.081032 26763 log.cc:1196] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/d89f2ded18f34fe88e5d860b31c377b0
I20260619 06:51:24.081612 26763 ts_tablet_manager.cc:1939] T d89f2ded18f34fe88e5d860b31c377b0 P 210e69b32f6a4b428c46fd0dad850087: Deleting consensus metadata
I20260619 06:51:24.082991 26173 catalog_manager.cc:4841] TS 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023): tablet d89f2ded18f34fe88e5d860b31c377b0 (table test-table [id=85803c1821c24758b0ecedd8631e135e]) successfully deleted
I20260619 06:51:24.083206 26762 ts_tablet_manager.cc:1918] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.083413 26762 log.cc:1196] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/d89f2ded18f34fe88e5d860b31c377b0
I20260619 06:51:24.083884 26762 ts_tablet_manager.cc:1939] T d89f2ded18f34fe88e5d860b31c377b0 P bb494569ce15479ab2d304c1e0d03028: Deleting consensus metadata
I20260619 06:51:24.085034 26173 catalog_manager.cc:4841] TS bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183): tablet d89f2ded18f34fe88e5d860b31c377b0 (table test-table [id=85803c1821c24758b0ecedd8631e135e]) successfully deleted
Jun 19 06:51:24 dist-test-slave-1h5s krb5kdc[26142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:51:24 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:24 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:24.134271 26736 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:24.122458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.18.39.126:42837)
0619 06:51:24.122676 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.122684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.123020 (+ 336us) server_negotiation.cc:407] Connection header received
0619 06:51:24.123235 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.123242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.123325 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.123446 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:24.124703 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.125894 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.127079 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.127722 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.127846 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.134021 (+ 6175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56370: BlockingRecv error: recv got EOF from 127.0.0.1:56370 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:51:24 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM
I20260619 06:51:24.152455 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.137099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.18.39.126:42837)
0619 06:51:24.137284 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.137290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.137881 (+ 591us) server_negotiation.cc:407] Connection header received
0619 06:51:24.138072 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.138078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.138152 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.138251 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:24.139416 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.140158 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.141473 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.142147 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.142289 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.146427 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:24.146434 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:24.146439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:24.146510 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:24.149991 (+ 3481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:24.150671 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:24.150677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:24.150681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:24.150824 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:24.151154 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:24.151159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:24.151162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:24.151480 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:24.151713 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.151899 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.152114 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:51:24.156086 26186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:24.156806 26186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:24.163828 26463 tablet_service.cc:1467] Processing CreateTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 (DEFAULT_TABLE table=test-table [id=0b91d9da760542cf8d8221c093a588ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.163843 26324 tablet_service.cc:1467] Processing CreateTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 (DEFAULT_TABLE table=test-table [id=0b91d9da760542cf8d8221c093a588ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.164268 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aabe2a481944b10a0ff4afb449be3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.164281 26324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aabe2a481944b10a0ff4afb449be3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.168644 26700 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Bootstrap starting.
I20260619 06:51:24.168711 26602 tablet_service.cc:1467] Processing CreateTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 (DEFAULT_TABLE table=test-table [id=0b91d9da760542cf8d8221c093a588ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.169127 26602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4aabe2a481944b10a0ff4afb449be3a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.170270 26700 tablet_bootstrap.cc:654] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.170796 26697 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Bootstrap starting.
I20260619 06:51:24.172436 26697 tablet_bootstrap.cc:654] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.178211 26698 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Bootstrap starting.
I20260619 06:51:24.180200 26698 tablet_bootstrap.cc:654] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.183508 26697 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: No bootstrap required, opened a new log
I20260619 06:51:24.183737 26697 ts_tablet_manager.cc:1397] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260619 06:51:24.184558 26697 raft_consensus.cc:348] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.184829 26697 raft_consensus.cc:374] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.184916 26697 raft_consensus.cc:729] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.185159 26697 consensus_queue.cc:260] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.186322 26700 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: No bootstrap required, opened a new log
I20260619 06:51:24.186506 26700 ts_tablet_manager.cc:1397] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.007s
I20260619 06:51:24.188145 26697 ts_tablet_manager.cc:1428] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 06:51:24.188936 26700 raft_consensus.cc:348] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.189357 26700 raft_consensus.cc:374] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.189467 26700 raft_consensus.cc:729] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.189867 26700 consensus_queue.cc:260] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.190541 26700 ts_tablet_manager.cc:1428] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260619 06:51:24.191617 26698 tablet_bootstrap.cc:492] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: No bootstrap required, opened a new log
I20260619 06:51:24.191833 26698 ts_tablet_manager.cc:1397] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260619 06:51:24.192644 26698 raft_consensus.cc:348] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.192893 26698 raft_consensus.cc:374] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.193007 26698 raft_consensus.cc:729] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.193264 26698 consensus_queue.cc:260] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.193848 26698 ts_tablet_manager.cc:1428] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:51:24.253796 26705 raft_consensus.cc:482] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:24.254055 26705 raft_consensus.cc:504] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.255803 26705 leader_election.cc:288] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023)
I20260619 06:51:24.275655 26781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.256143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:45183 (local address 127.18.39.66:51731)
0619 06:51:24.268297 (+ 12154us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:24.268326 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:24.268490 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:24.269055 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:24.269061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:24.269088 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:24.269394 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:24.269406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.270930 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.270936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:24.273108 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:24.273123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.274798 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.274804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:24.274980 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.275056 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 06:51:24.275124 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 06:51:24.275273 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11948,"thread_start_us":11808,"threads_started":1}
I20260619 06:51:24.276749 26780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.256213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:51731 (local address 127.18.39.65:45183)
0619 06:51:24.268647 (+ 12434us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.268654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.268674 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:51:24.268749 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.268755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.268825 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.269552 (+ 727us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:24.269718 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.270763 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.273323 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.274573 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.276164 (+ 1591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.276238 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.276289 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.276427 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12295,"thread_start_us":12686,"threads_started":1}
I20260619 06:51:24.277513 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aabe2a481944b10a0ff4afb449be3a7" candidate_uuid: "7e700c4a588d43139c79516c4622e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028" is_pre_election: true
I20260619 06:51:24.277806 26343 raft_consensus.cc:2454] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e700c4a588d43139c79516c4622e495 in term 0.
I20260619 06:51:24.278419 26416 leader_election.cc:302] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e700c4a588d43139c79516c4622e495, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:24.278894 26705 raft_consensus.cc:2781] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:24.279017 26705 raft_consensus.cc:482] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:24.279110 26705 raft_consensus.cc:3037] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.282821 26705 raft_consensus.cc:504] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.283663 26705 leader_election.cc:288] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [CANDIDATE]: Term 1 election: Requested vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023)
I20260619 06:51:24.284473 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aabe2a481944b10a0ff4afb449be3a7" candidate_uuid: "7e700c4a588d43139c79516c4622e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028"
I20260619 06:51:24.284713 26343 raft_consensus.cc:3037] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.285393 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.268598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:57639 (local address 127.18.39.67:43023)
0619 06:51:24.268863 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.268869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.275840 (+ 6971us) server_negotiation.cc:407] Connection header received
0619 06:51:24.276435 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.276441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.276520 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.276587 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:24.278019 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.279108 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.283007 (+ 3899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.284354 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.284516 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.284610 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.284970 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.285108 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:51:24.285429 26782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.269036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43023 (local address 127.18.39.66:57639)
0619 06:51:24.275710 (+ 6674us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:24.275734 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:24.276263 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:24.276814 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:24.276819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:24.276847 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:24.277216 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:24.277230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.279341 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.279346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:24.282817 (+ 3471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:24.282830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.284535 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.284541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:24.284772 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.284828 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:51:24.284933 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:51:24.285084 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6514,"thread_start_us":6808,"threads_started":1}
I20260619 06:51:24.286206 26624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aabe2a481944b10a0ff4afb449be3a7" candidate_uuid: "7e700c4a588d43139c79516c4622e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210e69b32f6a4b428c46fd0dad850087" is_pre_election: true
I20260619 06:51:24.286453 26624 raft_consensus.cc:2454] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e700c4a588d43139c79516c4622e495 in term 0.
I20260619 06:51:24.286424 26621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4aabe2a481944b10a0ff4afb449be3a7" candidate_uuid: "7e700c4a588d43139c79516c4622e495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210e69b32f6a4b428c46fd0dad850087"
I20260619 06:51:24.286859 26621 raft_consensus.cc:3037] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.287825 26343 raft_consensus.cc:2454] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e700c4a588d43139c79516c4622e495 in term 1.
I20260619 06:51:24.288307 26416 leader_election.cc:302] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e700c4a588d43139c79516c4622e495, bb494569ce15479ab2d304c1e0d03028; no voters:
I20260619 06:51:24.288596 26705 raft_consensus.cc:2781] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:24.289062 26705 raft_consensus.cc:686] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 1 LEADER]: Becoming Leader. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Running, Role: LEADER
I20260619 06:51:24.289496 26705 consensus_queue.cc:237] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.290033 26621 raft_consensus.cc:2454] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e700c4a588d43139c79516c4622e495 in term 1.
I20260619 06:51:24.293229 26186 catalog_manager.cc:5495] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e700c4a588d43139c79516c4622e495 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "7e700c4a588d43139c79516c4622e495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } health_report { overall_health: UNKNOWN } } }
I20260619 06:51:24.317390 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.310593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.18.39.66:43041)
0619 06:51:24.311259 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.311265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.311308 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:51:24.311420 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.311426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.311538 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.311724 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:24.312070 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.313264 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.314599 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.315300 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.315433 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.315866 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:24.316686 (+ 820us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:24.316850 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.316899 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.316978 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":385,"threads_started":1}
I20260619 06:51:24.321484 26621 raft_consensus.cc:1261] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Refusing update from remote peer 7e700c4a588d43139c79516c4622e495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:24.321513 26343 raft_consensus.cc:1261] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Refusing update from remote peer 7e700c4a588d43139c79516c4622e495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:24.322149 26783 consensus_queue.cc:1035] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:24.322594 26705 consensus_queue.cc:1035] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:24.330878 26747 mvcc.cc:205] Tried to move back new op lower bound from 7298465318173417472 to 7298465318054518784. Current Snapshot: MvccSnapshot[applied={T|T < 7298465318173417472}]
I20260619 06:51:24.346207 26186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.346495 26186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.350574 26186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Sending DeleteTablet for 3 replicas of tablet 4aabe2a481944b10a0ff4afb449be3a7
I20260619 06:51:24.351522 26324 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:39732
I20260619 06:51:24.351642 26463 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42574
I20260619 06:51:24.351822 26762 tablet_replica.cc:330] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: stopping tablet replica
I20260619 06:51:24.351922 26764 tablet_replica.cc:330] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: stopping tablet replica
I20260619 06:51:24.352020 26762 raft_consensus.cc:2229] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.352114 26764 raft_consensus.cc:2229] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:24.352274 26762 raft_consensus.cc:2258] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.352632 26764 raft_consensus.cc:2258] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.353734 26764 ts_tablet_manager.cc:1905] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.353767 26762 ts_tablet_manager.cc:1905] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.360703 26762 ts_tablet_manager.cc:1918] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:24.360726 26764 ts_tablet_manager.cc:1918] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:24.360864 26762 log.cc:1196] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/4aabe2a481944b10a0ff4afb449be3a7
I20260619 06:51:24.360879 26764 log.cc:1196] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/4aabe2a481944b10a0ff4afb449be3a7
I20260619 06:51:24.361594 26762 ts_tablet_manager.cc:1939] T 4aabe2a481944b10a0ff4afb449be3a7 P bb494569ce15479ab2d304c1e0d03028: Deleting consensus metadata
I20260619 06:51:24.362927 26173 catalog_manager.cc:4841] TS bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183): tablet 4aabe2a481944b10a0ff4afb449be3a7 (table test-table [id=0b91d9da760542cf8d8221c093a588ec]) successfully deleted
I20260619 06:51:24.366360 26602 tablet_service.cc:1514] Processing DeleteTablet for tablet 4aabe2a481944b10a0ff4afb449be3a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:32774
I20260619 06:51:24.366746 26763 tablet_replica.cc:330] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: stopping tablet replica
I20260619 06:51:24.366963 26763 raft_consensus.cc:2229] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.367215 26763 raft_consensus.cc:2258] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.367378 26185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:24.368064 26185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:24.368631 26763 ts_tablet_manager.cc:1905] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.370070 26764 ts_tablet_manager.cc:1939] T 4aabe2a481944b10a0ff4afb449be3a7 P 7e700c4a588d43139c79516c4622e495: Deleting consensus metadata
I20260619 06:51:24.371495 26172 catalog_manager.cc:4841] TS 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041): tablet 4aabe2a481944b10a0ff4afb449be3a7 (table test-table [id=0b91d9da760542cf8d8221c093a588ec]) successfully deleted
I20260619 06:51:24.376716 26763 ts_tablet_manager.cc:1918] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:24.377938 26763 log.cc:1196] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/4aabe2a481944b10a0ff4afb449be3a7
I20260619 06:51:24.378518 26763 ts_tablet_manager.cc:1939] T 4aabe2a481944b10a0ff4afb449be3a7 P 210e69b32f6a4b428c46fd0dad850087: Deleting consensus metadata
I20260619 06:51:24.379720 26173 catalog_manager.cc:4841] TS 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023): tablet 4aabe2a481944b10a0ff4afb449be3a7 (table test-table [id=0b91d9da760542cf8d8221c093a588ec]) successfully deleted
I20260619 06:51:24.383754 26324 tablet_service.cc:1467] Processing CreateTablet for tablet 826aa165f8fb4180889c59ec017f49f0 (DEFAULT_TABLE table=test-table [id=9804ff3fd1e64ef7be4a4532de14b531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.384258 26324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826aa165f8fb4180889c59ec017f49f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.386106 26602 tablet_service.cc:1467] Processing CreateTablet for tablet 826aa165f8fb4180889c59ec017f49f0 (DEFAULT_TABLE table=test-table [id=9804ff3fd1e64ef7be4a4532de14b531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.386565 26602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826aa165f8fb4180889c59ec017f49f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.388200 26463 tablet_service.cc:1467] Processing CreateTablet for tablet 826aa165f8fb4180889c59ec017f49f0 (DEFAULT_TABLE table=test-table [id=9804ff3fd1e64ef7be4a4532de14b531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:24.388665 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 826aa165f8fb4180889c59ec017f49f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:24.397775 26700 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Bootstrap starting.
I20260619 06:51:24.399405 26700 tablet_bootstrap.cc:654] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.408528 26697 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Bootstrap starting.
I20260619 06:51:24.410319 26697 tablet_bootstrap.cc:654] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.412817 26700 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: No bootstrap required, opened a new log
I20260619 06:51:24.413010 26700 ts_tablet_manager.cc:1397] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.007s
I20260619 06:51:24.415315 26698 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Bootstrap starting.
I20260619 06:51:24.415261 26700 raft_consensus.cc:348] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.415512 26700 raft_consensus.cc:374] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.415583 26700 raft_consensus.cc:729] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e700c4a588d43139c79516c4622e495, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.415910 26700 consensus_queue.cc:260] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.416433 26700 ts_tablet_manager.cc:1428] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:24.417088 26698 tablet_bootstrap.cc:654] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:24.423599 26698 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: No bootstrap required, opened a new log
I20260619 06:51:24.423818 26698 ts_tablet_manager.cc:1397] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260619 06:51:24.424690 26698 raft_consensus.cc:348] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.424995 26698 raft_consensus.cc:374] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.425081 26698 raft_consensus.cc:729] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.425408 26698 consensus_queue.cc:260] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.428387 26698 ts_tablet_manager.cc:1428] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260619 06:51:24.431690 26697 tablet_bootstrap.cc:492] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: No bootstrap required, opened a new log
I20260619 06:51:24.431924 26697 ts_tablet_manager.cc:1397] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.003s
I20260619 06:51:24.432685 26697 raft_consensus.cc:348] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.432994 26697 raft_consensus.cc:374] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:24.433070 26697 raft_consensus.cc:729] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb494569ce15479ab2d304c1e0d03028, State: Initialized, Role: FOLLOWER
I20260619 06:51:24.433305 26697 consensus_queue.cc:260] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.435288 26697 ts_tablet_manager.cc:1428] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:24.566104 26728 raft_consensus.cc:482] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:24.566375 26728 raft_consensus.cc:504] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.567118 26728 leader_election.cc:288] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:24.567749 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826aa165f8fb4180889c59ec017f49f0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028" is_pre_election: true
I20260619 06:51:24.567859 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826aa165f8fb4180889c59ec017f49f0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495" is_pre_election: true
I20260619 06:51:24.568015 26343 raft_consensus.cc:2454] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:24.568089 26483 raft_consensus.cc:2454] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 0.
I20260619 06:51:24.568588 26554 leader_election.cc:302] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, 7e700c4a588d43139c79516c4622e495; no voters:
I20260619 06:51:24.568910 26728 raft_consensus.cc:2781] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:24.569015 26728 raft_consensus.cc:482] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:24.569114 26728 raft_consensus.cc:3037] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.572259 26728 raft_consensus.cc:504] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.573544 26483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826aa165f8fb4180889c59ec017f49f0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e700c4a588d43139c79516c4622e495"
I20260619 06:51:24.573566 26343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "826aa165f8fb4180889c59ec017f49f0" candidate_uuid: "210e69b32f6a4b428c46fd0dad850087" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb494569ce15479ab2d304c1e0d03028"
I20260619 06:51:24.573758 26483 raft_consensus.cc:3037] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.573794 26343 raft_consensus.cc:3037] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:24.576738 26343 raft_consensus.cc:2454] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:24.576761 26483 raft_consensus.cc:2454] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210e69b32f6a4b428c46fd0dad850087 in term 1.
I20260619 06:51:24.577265 26554 leader_election.cc:302] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210e69b32f6a4b428c46fd0dad850087, 7e700c4a588d43139c79516c4622e495; no voters:
I20260619 06:51:24.577704 26728 leader_election.cc:288] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [CANDIDATE]: Term 1 election: Requested vote from peers bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183), 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041)
I20260619 06:51:24.577948 26728 raft_consensus.cc:2781] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:24.578159 26728 raft_consensus.cc:686] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 LEADER]: Becoming Leader. State: Replica: 210e69b32f6a4b428c46fd0dad850087, State: Running, Role: LEADER
I20260619 06:51:24.578496 26728 consensus_queue.cc:237] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } }
I20260619 06:51:24.589367 26186 catalog_manager.cc:5495] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 reported cstate change: term changed from 0 to 1, leader changed from <none> to 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "210e69b32f6a4b428c46fd0dad850087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210e69b32f6a4b428c46fd0dad850087" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43023 } health_report { overall_health: HEALTHY } } }
I20260619 06:51:24.663801 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.656931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.18.39.67:43023)
0619 06:51:24.657133 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.657139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.657158 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:51:24.657227 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.657232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.657369 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.657456 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:24.658054 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.659218 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.661215 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.661960 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.662085 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.662751 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:24.662896 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:24.662991 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.663158 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.663244 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:51:24.676937 26483 raft_consensus.cc:1261] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:24.677157 26343 raft_consensus.cc:1261] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Refusing update from remote peer 210e69b32f6a4b428c46fd0dad850087: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:24.677770 26796 consensus_queue.cc:1035] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e700c4a588d43139c79516c4622e495" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:24.678141 26797 consensus_queue.cc:1035] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb494569ce15479ab2d304c1e0d03028" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 45183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:24.804807 26186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.805066 26186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:24.810911 26186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d6cebc734e14bb396052f947becc755: Sending DeleteTablet for 3 replicas of tablet 826aa165f8fb4180889c59ec017f49f0
I20260619 06:51:24.812036 26324 tablet_service.cc:1514] Processing DeleteTablet for tablet 826aa165f8fb4180889c59ec017f49f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:39732
I20260619 06:51:24.812232 26463 tablet_service.cc:1514] Processing DeleteTablet for tablet 826aa165f8fb4180889c59ec017f49f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42574
I20260619 06:51:24.812348 26762 tablet_replica.cc:330] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: stopping tablet replica
I20260619 06:51:24.812510 26764 tablet_replica.cc:330] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: stopping tablet replica
I20260619 06:51:24.812521 26762 raft_consensus.cc:2229] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.812688 26764 raft_consensus.cc:2229] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:24.812731 26762 raft_consensus.cc:2258] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.812896 26764 raft_consensus.cc:2258] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.814025 26762 ts_tablet_manager.cc:1905] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.814399 26601 tablet_service.cc:1514] Processing DeleteTablet for tablet 826aa165f8fb4180889c59ec017f49f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:24 UTC) from {username='oryx', principal='oryx/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:32774
I20260619 06:51:24.815855 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26258
I20260619 06:51:24.817410 26763 tablet_replica.cc:330] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: stopping tablet replica
I20260619 06:51:24.817626 26763 raft_consensus.cc:2229] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:24.818041 26763 raft_consensus.cc:2258] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:24.818863 26764 ts_tablet_manager.cc:1905] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.822969 26763 ts_tablet_manager.cc:1905] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:24.854039 26762 ts_tablet_manager.cc:1918] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.854251 26762 log.cc:1196] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/826aa165f8fb4180889c59ec017f49f0
I20260619 06:51:24.854801 26762 ts_tablet_manager.cc:1939] T 826aa165f8fb4180889c59ec017f49f0 P bb494569ce15479ab2d304c1e0d03028: Deleting consensus metadata
I20260619 06:51:24.856110 26764 ts_tablet_manager.cc:1918] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.856463 26764 log.cc:1196] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/826aa165f8fb4180889c59ec017f49f0
I20260619 06:51:24.857021 26764 ts_tablet_manager.cc:1939] T 826aa165f8fb4180889c59ec017f49f0 P 7e700c4a588d43139c79516c4622e495: Deleting consensus metadata
I20260619 06:51:24.858668 26172 catalog_manager.cc:4841] TS 7e700c4a588d43139c79516c4622e495 (127.18.39.66:43041): tablet 826aa165f8fb4180889c59ec017f49f0 (table test-table [id=9804ff3fd1e64ef7be4a4532de14b531]) successfully deleted
I20260619 06:51:24.860718 26173 catalog_manager.cc:4841] TS bb494569ce15479ab2d304c1e0d03028 (127.18.39.65:45183): tablet 826aa165f8fb4180889c59ec017f49f0 (table test-table [id=9804ff3fd1e64ef7be4a4532de14b531]) successfully deleted
I20260619 06:51:24.861227 26763 ts_tablet_manager.cc:1918] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:51:24.861413 26763 log.cc:1196] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/826aa165f8fb4180889c59ec017f49f0
I20260619 06:51:24.862054 26763 ts_tablet_manager.cc:1939] T 826aa165f8fb4180889c59ec017f49f0 P 210e69b32f6a4b428c46fd0dad850087: Deleting consensus metadata
I20260619 06:51:24.864399 26173 catalog_manager.cc:4841] TS 210e69b32f6a4b428c46fd0dad850087 (127.18.39.67:43023): tablet 826aa165f8fb4180889c59ec017f49f0 (table test-table [id=9804ff3fd1e64ef7be4a4532de14b531]) successfully deleted
Jun 19 06:51:24 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.65@KRBTEST.COM
I20260619 06:51:24.877799 26803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:24.821551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.18.39.65:45183)
0619 06:51:24.827238 (+ 5687us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:24.827247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:24.859497 (+ 32250us) server_negotiation.cc:407] Connection header received
0619 06:51:24.859703 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:24.859712 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:24.859803 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:24.859908 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:24.861785 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.862944 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.864692 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:24.865368 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:24.865469 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:24.870580 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:24.870601 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:24.870624 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:24.870682 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:24.874478 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:24.875310 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:24.875319 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:24.875323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:24.875491 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:24.875914 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:24.875919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:24.875923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:24.876360 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:24.876638 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:24.877177 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 06:51:24.877434 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5519,"thread_start_us":6745,"threads_started":1}
I20260619 06:51:24.879160 26389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:25.018998 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26258
I20260619 06:51:25.041268 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26398
Jun 19 06:51:25 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.66@KRBTEST.COM
I20260619 06:51:25.056365 26805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:25.042198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.18.39.66:43041)
0619 06:51:25.042717 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:25.042722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:25.042738 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:51:25.042791 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:25.042795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:25.042861 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:25.042953 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:25.044411 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:25.045114 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:25.046501 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:25.046905 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:25.047022 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:25.051197 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:25.051212 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:25.051228 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:25.051263 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:25.054061 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:25.054691 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:25.054698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:25.054700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:25.054825 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:25.055138 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:25.055141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:25.055143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:25.055489 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:25.055700 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:25.055931 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 06:51:25.056095 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":304,"threads_started":1}
I20260619 06:51:25.057054 26528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:25.193645 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26398
I20260619 06:51:25.215741 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26537
Jun 19 06:51:25 dist-test-slave-1h5s krb5kdc[26142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.39.67@KRBTEST.COM
I20260619 06:51:25.230914 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:25.216592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.18.39.67:43023)
0619 06:51:25.217035 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:25.217039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:25.217054 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:51:25.217105 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:25.217109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:25.217164 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:25.217277 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:25.218651 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:25.219381 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:25.220602 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:25.221002 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:25.221140 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:25.225368 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:25.225412 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:25.225428 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:25.225472 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:25.228081 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:25.228762 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:25.228771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:25.228780 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:25.228906 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:25.229260 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:25.229266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:25.229270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:25.229724 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:25.230016 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:25.230328 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 06:51:25.230567 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":227,"threads_started":1}
I20260619 06:51:25.231599 26667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:25.377261 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26537
I20260619 06:51:25.402282 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26155
I20260619 06:51:25.403466 26227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:25.518357 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26155
2026-06-19T06:51:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7193 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:25 dist-test-slave-1h5s krb5kdc[26814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:25 dist-test-slave-1h5s krb5kdc[26814](info): set up 2 sockets
Jun 19 06:51:25 dist-test-slave-1h5s krb5kdc[26814](info): commencing operation
krb5kdc: starting...
W20260619 06:51:26.614301 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:26 dist-test-slave-1h5s krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:51:26.834476 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46467
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:46467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:27.167636 26827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:27.167932 26827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:27.168078 26827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:27.175388 26827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:27.175509 26827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:27.175601 26827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:27.175698 26827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:27.175776 26827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:27.185909 26827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:46467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:27.187278 26827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:27.188673 26827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:27.200780 26833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:27.200755 26832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:27.200927 26835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:27.201687 26827 server_base.cc:1034] running on GCE node
I20260619 06:51:27.202486 26827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:27.203958 26827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:27.205186 26827 hybrid_clock.cc:648] HybridClock initialized: now 1781851887205136 us; error 41 us; skew 500 ppm
I20260619 06:51:27.205575 26827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T06:51:27Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1752 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:27 dist-test-slave-1h5s krb5kdc[26841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:27 dist-test-slave-1h5s krb5kdc[26841](info): set up 2 sockets
Jun 19 06:51:27 dist-test-slave-1h5s krb5kdc[26841](info): commencing operation
krb5kdc: starting...
W20260619 06:51:28.386246 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.007s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:28 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:51:28.610945 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38801
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:38801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:28.935842 26854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:28.936148 26854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:28.936302 26854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:28.943621 26854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:28.943734 26854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:28.943779 26854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:28.943827 26854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:28.943915 26854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:28.953935 26854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:38801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:28.955277 26854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:28.956562 26854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:28.968427 26860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:28.968406 26862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:28.968719 26859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:28.969383 26854 server_base.cc:1034] running on GCE node
I20260619 06:51:28.970245 26854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:28.971751 26854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:28.972973 26854 hybrid_clock.cc:648] HybridClock initialized: now 1781851888972897 us; error 60 us; skew 500 ppm
I20260619 06:51:28.973439 26854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:28 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:28.979691 26854 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:51:28.981372 26854 webserver.cc:460] Webserver started at http://127.18.39.126:44253/ using document root <none> and password file <none>
I20260619 06:51:28.982178 26854 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:28.982518 26854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:28.982872 26854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:28.985886 26854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "678e454d43004459ae976bb60a20de94"
format_stamp: "Formatted at 2026-06-19 06:51:28 on dist-test-slave-1h5s"
server_key: "1b73c19484d9985bdc747f79efc3bc62"
server_key_iv: "72c0e16370ec86af9e89dc3c4e3b6097"
server_key_version: "encryptionkey@0"
I20260619 06:51:28.986632 26854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "678e454d43004459ae976bb60a20de94"
format_stamp: "Formatted at 2026-06-19 06:51:28 on dist-test-slave-1h5s"
server_key: "1b73c19484d9985bdc747f79efc3bc62"
server_key_iv: "72c0e16370ec86af9e89dc3c4e3b6097"
server_key_version: "encryptionkey@0"
I20260619 06:51:28.991796 26854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:28.995453 26869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:28.996886 26854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:28.997097 26854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "678e454d43004459ae976bb60a20de94"
format_stamp: "Formatted at 2026-06-19 06:51:28 on dist-test-slave-1h5s"
server_key: "1b73c19484d9985bdc747f79efc3bc62"
server_key_iv: "72c0e16370ec86af9e89dc3c4e3b6097"
server_key_version: "encryptionkey@0"
I20260619 06:51:28.997315 26854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:29.024420 26854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:29.028746 26854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:29.029053 26854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:29.047914 26854 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:38801
I20260619 06:51:29.047937 26920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:38801 every 8 connection(s)
I20260619 06:51:29.049322 26854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:29.052616 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26854
I20260619 06:51:29.052839 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:29.053210 26921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:29.053560 18589 external_mini_cluster.cc:1428] Setting key 3159ebbeaef3b271f65e5553c5e99648
I20260619 06:51:29.068434 26921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Bootstrap starting.
Jun 19 06:51:29 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:29.072671 26921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:29.074101 26921 log.cc:824] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:29.077883 26921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: No bootstrap required, opened a new log
I20260619 06:51:29.078531 26924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:29.055611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.18.39.126:38801)
0619 06:51:29.056494 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:29.056515 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:29.056568 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:51:29.058441 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:29.058474 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:29.059065 (+ 591us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:29.059572 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:29.060951 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.062963 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.064217 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.065383 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.065589 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:29.069645 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:29.069656 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:29.069676 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:29.069725 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:29.073761 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:29.074756 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:29.074770 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:29.074780 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:29.075058 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:29.075523 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:29.075527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:29.075529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:29.076232 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:29.076545 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:29.076942 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 06:51:29.077472 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":365,"threads_started":1}
I20260619 06:51:29.085659 26921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:29.086088 26921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:29.086223 26921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678e454d43004459ae976bb60a20de94, State: Initialized, Role: FOLLOWER
I20260619 06:51:29.086931 26921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:29.087152 26921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:29.087252 26921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:29.087376 26921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:29.089584 26921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:29.090082 26921 leader_election.cc:302] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 678e454d43004459ae976bb60a20de94; no voters:
I20260619 06:51:29.090643 26921 leader_election.cc:288] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:29.090808 26926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:29.091542 26926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 LEADER]: Becoming Leader. State: Replica: 678e454d43004459ae976bb60a20de94, State: Running, Role: LEADER
I20260619 06:51:29.092219 26926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:29.092819 26921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:29.095196 26927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "678e454d43004459ae976bb60a20de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } } }
I20260619 06:51:29.095646 26927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:29.095221 26928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678e454d43004459ae976bb60a20de94. Latest consensus state: current_term: 1 leader_uuid: "678e454d43004459ae976bb60a20de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } } }
I20260619 06:51:29.097638 26928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:29.098165 26933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:29.103605 26933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:29.112789 26933 catalog_manager.cc:1347] Generated new cluster ID: 75783a99fec842e6b6ce04b148c6c1c1
I20260619 06:51:29.112920 26933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:29.129313 26933 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:29.130702 26933 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:29.142570 26933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Generated new TSK 0
I20260619 06:51:29.143440 26933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:51:29.204797 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:29.551589 26947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:29.551885 26947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:29.552062 26947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:29.559391 26947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:29.559497 26947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:29.559666 26947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:29.569465 26947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:29.570730 26947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:29.572026 26947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:29.585305 26952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:29.585260 26953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:29.585384 26955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:29.590967 26947 server_base.cc:1034] running on GCE node
I20260619 06:51:29.591652 26947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:29.592648 26947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:29.593873 26947 hybrid_clock.cc:648] HybridClock initialized: now 1781851889593811 us; error 76 us; skew 500 ppm
I20260619 06:51:29.594242 26947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:29 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:29.597412 26947 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:51:29.599216 26947 webserver.cc:460] Webserver started at http://127.18.39.65:42183/ using document root <none> and password file <none>
I20260619 06:51:29.599949 26947 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:29.600095 26947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:29.600414 26947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:29.603526 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "b8d30b76e3a14d74bd528992a13c1544"
format_stamp: "Formatted at 2026-06-19 06:51:29 on dist-test-slave-1h5s"
server_key: "b93e24d6c1c04ea6d68a8c6b28d55d66"
server_key_iv: "066c55e7cd69be2f88a25e9fc39fc9be"
server_key_version: "encryptionkey@0"
I20260619 06:51:29.604393 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "b8d30b76e3a14d74bd528992a13c1544"
format_stamp: "Formatted at 2026-06-19 06:51:29 on dist-test-slave-1h5s"
server_key: "b93e24d6c1c04ea6d68a8c6b28d55d66"
server_key_iv: "066c55e7cd69be2f88a25e9fc39fc9be"
server_key_version: "encryptionkey@0"
I20260619 06:51:29.609987 26947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:51:29.613595 26962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:29.615100 26947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:29.615257 26947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "b8d30b76e3a14d74bd528992a13c1544"
format_stamp: "Formatted at 2026-06-19 06:51:29 on dist-test-slave-1h5s"
server_key: "b93e24d6c1c04ea6d68a8c6b28d55d66"
server_key_iv: "066c55e7cd69be2f88a25e9fc39fc9be"
server_key_version: "encryptionkey@0"
I20260619 06:51:29.615442 26947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:29.631779 26947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:29.635931 26947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:29.636181 26947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:29.637195 26947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:29.638628 26947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:29.638705 26947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:29.638790 26947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:29.638834 26947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:29.672307 26947 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:46517
I20260619 06:51:29.672350 27074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:46517 every 8 connection(s)
I20260619 06:51:29.673688 26947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:29.681118 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 26947
I20260619 06:51:29.681354 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:29.682049 18589 external_mini_cluster.cc:1428] Setting key 93140efcebea648cfca0a64102ff774c
Jun 19 06:51:29 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851889, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:29.694954 27077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:29.676147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:38851 (local address 127.18.39.126:38801)
0619 06:51:29.676589 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:29.676593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:29.677666 (+ 1073us) server_negotiation.cc:407] Connection header received
0619 06:51:29.679572 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:29.679577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:29.679669 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:29.679788 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:29.681642 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.682479 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.684421 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.684842 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.684954 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:29.689128 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:29.689133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:29.689137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:29.689169 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:29.691721 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:29.692468 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:29.692477 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:29.692479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:29.692587 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:29.693076 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:29.693083 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:29.693087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:29.693431 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:29.693581 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:29.694360 (+ 779us) server_negotiation.cc:299] Negotiation successful
0619 06:51:29.694576 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260619 06:51:29.695930 27078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:29.676754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:38851)
0619 06:51:29.677511 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:29.677549 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:29.679285 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:29.680027 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:29.680042 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:29.680551 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:29.681453 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:29.681479 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.682664 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.682669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:29.684176 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:29.684191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:29.685085 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:29.685094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:29.685466 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:29.686228 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:29.686264 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:29.688896 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:29.691868 (+ 2972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:29.691878 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:29.691898 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:29.692330 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:29.692697 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:29.692702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:29.692706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:29.692963 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:29.693554 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:29.693563 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:29.693978 (+ 415us) client_negotiation.cc:769] Sending connection context
0619 06:51:29.694253 (+ 275us) client_negotiation.cc:240] Negotiation successful
0619 06:51:29.694821 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":263,"threads_started":1}
I20260619 06:51:29.697629 27075 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38801
I20260619 06:51:29.697949 27075 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:29.698887 27075 heartbeater.cc:511] Master 127.18.39.126:38801 requested a full tablet report, sending...
I20260619 06:51:29.701448 26886 ts_manager.cc:194] Registered new tserver with Master: b8d30b76e3a14d74bd528992a13c1544 (127.18.39.65:46517)
I20260619 06:51:29.703686 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:38851
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:51:29.732613 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:30.062494 27081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:30.062791 27081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:30.062959 27081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:30.070377 27081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:30.070505 27081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:30.070719 27081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:30.080883 27081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:30.082278 27081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:30.083642 27081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:30.097223 27087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:30.097256 27086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:30.097781 27089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:30.098066 27081 server_base.cc:1034] running on GCE node
I20260619 06:51:30.098850 27081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:30.099802 27081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:30.100993 27081 hybrid_clock.cc:648] HybridClock initialized: now 1781851890100959 us; error 35 us; skew 500 ppm
I20260619 06:51:30.101383 27081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:30 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:30.104503 27081 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:51:30.106079 27081 webserver.cc:460] Webserver started at http://127.18.39.66:43679/ using document root <none> and password file <none>
I20260619 06:51:30.106814 27081 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:30.106959 27081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:30.107268 27081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:30.110249 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "3f3468c38a8442c4b81c17f291e80793"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "9be53f855525126398bb420dd9cf1777"
server_key_iv: "f4b0d2cc4bb7d6f98b09d9fee299432c"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.111052 27081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "3f3468c38a8442c4b81c17f291e80793"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "9be53f855525126398bb420dd9cf1777"
server_key_iv: "f4b0d2cc4bb7d6f98b09d9fee299432c"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.116370 27081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:51:30.120011 27096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.121449 27081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:30.121622 27081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "3f3468c38a8442c4b81c17f291e80793"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "9be53f855525126398bb420dd9cf1777"
server_key_iv: "f4b0d2cc4bb7d6f98b09d9fee299432c"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.121824 27081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:30.162803 27081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:30.167407 27081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:30.167727 27081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:30.168692 27081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:30.170151 27081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:30.170251 27081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.170403 27081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:30.170475 27081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.203759 27081 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:39443
I20260619 06:51:30.203805 27208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:39443 every 8 connection(s)
I20260619 06:51:30.205600 27081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:30.214618 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27081
I20260619 06:51:30.214828 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:30.215307 18589 external_mini_cluster.cc:1428] Setting key b1cf15af7f0f3849b2916827f3e53d5d
Jun 19 06:51:30 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:30.227803 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:30.208015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:46591 (local address 127.18.39.126:38801)
0619 06:51:30.208441 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:30.208446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:30.209421 (+ 975us) server_negotiation.cc:407] Connection header received
0619 06:51:30.211242 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:30.211247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:30.211358 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:30.211474 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:30.212948 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.213729 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.215976 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.216510 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.216645 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:30.220296 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:30.220305 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:30.220313 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:30.220357 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:30.224206 (+ 3849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.225186 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.225191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.225193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.225323 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.225895 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.225901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.225906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.226205 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:30.226354 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:30.227190 (+ 836us) server_negotiation.cc:299] Negotiation successful
0619 06:51:30.227415 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":237,"threads_started":1}
I20260619 06:51:30.228842 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:30.208532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:46591)
0619 06:51:30.209196 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:30.209245 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:30.210969 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:30.211705 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:30.211714 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:30.212219 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:30.212781 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:30.212797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.213871 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.213876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:30.215802 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:30.215815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.216747 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.216752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:30.216952 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:30.217737 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:30.217760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:30.220038 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:30.224383 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:30.224390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:30.224406 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:30.225062 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:30.225500 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:30.225505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:30.225509 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:30.225784 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:30.226346 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:30.226354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:30.226734 (+ 380us) client_negotiation.cc:769] Sending connection context
0619 06:51:30.227086 (+ 352us) client_negotiation.cc:240] Negotiation successful
0619 06:51:30.227645 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":265,"threads_started":1}
I20260619 06:51:30.230319 27209 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38801
I20260619 06:51:30.230664 27209 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:30.231561 27209 heartbeater.cc:511] Master 127.18.39.126:38801 requested a full tablet report, sending...
I20260619 06:51:30.233270 26886 ts_manager.cc:194] Registered new tserver with Master: 3f3468c38a8442c4b81c17f291e80793 (127.18.39.66:39443)
I20260619 06:51:30.234602 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:46591
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:51:30.268167 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:30.604163 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:30.604470 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:30.604625 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:30.612043 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:30.612160 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:30.612370 27215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:30.622689 27215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:30.624025 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:30.625541 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:30.638907 27223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:30.638953 27221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:30.639215 27220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:30.640538 27215 server_base.cc:1034] running on GCE node
I20260619 06:51:30.641145 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:30.642170 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:30.643437 27215 hybrid_clock.cc:648] HybridClock initialized: now 1781851890643385 us; error 45 us; skew 500 ppm
I20260619 06:51:30.643754 27215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:30 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:30.646886 27215 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:51:30.648356 27215 webserver.cc:460] Webserver started at http://127.18.39.67:40379/ using document root <none> and password file <none>
I20260619 06:51:30.649103 27215 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:30.649201 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:30.649529 27215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:30.652395 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "28acff3d075443939ce38ea5afe2103c"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "b537bfbbfff0867aeaa53b92bcede4d5"
server_key_iv: "17f28465b62b7d18afa7889b36d008b7"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.653110 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "28acff3d075443939ce38ea5afe2103c"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "b537bfbbfff0867aeaa53b92bcede4d5"
server_key_iv: "17f28465b62b7d18afa7889b36d008b7"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.658376 27215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:51:30.662088 27230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.663547 27215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:30.663690 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "28acff3d075443939ce38ea5afe2103c"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "b537bfbbfff0867aeaa53b92bcede4d5"
server_key_iv: "17f28465b62b7d18afa7889b36d008b7"
server_key_version: "encryptionkey@0"
I20260619 06:51:30.663877 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:30.690032 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:30.694415 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:30.694667 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:30.695591 27215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:30.696894 27215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:30.696967 27215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.697050 27215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:30.697094 27215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:30.707134 27075 heartbeater.cc:503] Master 127.18.39.126:38801 was elected leader, sending a full tablet report...
I20260619 06:51:30.731060 27215 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:38227
I20260619 06:51:30.731148 27342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:38227 every 8 connection(s)
I20260619 06:51:30.732383 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:30.741060 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27215
I20260619 06:51:30.741266 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:30.741927 18589 external_mini_cluster.cc:1428] Setting key 9f1d9591d5daac50c08f11b896c7ceff
Jun 19 06:51:30 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851890, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:30.752710 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:30.734886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:45505 (local address 127.18.39.126:38801)
0619 06:51:30.735362 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:30.735369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:30.736319 (+ 950us) server_negotiation.cc:407] Connection header received
0619 06:51:30.738179 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:30.738184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:30.738274 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:30.738388 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:30.739839 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.740622 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.743002 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.743587 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.743688 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:30.747485 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:30.747491 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:30.747494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:30.747528 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:30.749972 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.750704 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.750708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.750710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.750836 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.751216 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.751219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.751221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.751487 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:30.751600 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:30.752272 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 06:51:30.752443 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":220,"threads_started":1}
I20260619 06:51:30.753775 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:30.735475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:45505)
0619 06:51:30.736153 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:30.736197 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:30.737909 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:30.738595 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:30.738604 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:30.739116 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:30.739687 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:30.739706 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.740774 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.740780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:30.742838 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:30.742852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.743834 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.743843 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:30.744139 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:30.744903 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:30.744936 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:30.747272 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:30.750112 (+ 2840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:30.750119 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:30.750134 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:30.750590 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:30.750926 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:30.750930 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:30.750932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:30.751108 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:30.751598 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:30.751604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:30.751946 (+ 342us) client_negotiation.cc:769] Sending connection context
0619 06:51:30.752175 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 06:51:30.752575 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":262,"threads_started":1}
I20260619 06:51:30.755149 27343 heartbeater.cc:347] Connected to a master server at 127.18.39.126:38801
I20260619 06:51:30.755455 27343 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:30.756386 27343 heartbeater.cc:511] Master 127.18.39.126:38801 requested a full tablet report, sending...
I20260619 06:51:30.758059 26886 ts_manager.cc:194] Registered new tserver with Master: 28acff3d075443939ce38ea5afe2103c (127.18.39.67:38227)
I20260619 06:51:30.758879 26886 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:45505
I20260619 06:51:30.770155 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
Entry for principal oryx/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab.
WARNING: no policy specified for oryx/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
Entry for principal oryx/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab.
WARNING: no policy specified for oryx/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
Entry for principal oryx/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab.
I20260619 06:51:30.903687 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26947
Jun 19 06:51:30 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:51:30.921121 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:30.904824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.18.39.65:46517)
0619 06:51:30.905583 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:30.905593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:30.905634 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:51:30.905698 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:30.905704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:30.905940 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:30.906121 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:30.907634 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.908665 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.910065 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:30.910687 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:30.910841 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:30.915363 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:30.915389 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:30.915407 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:30.915450 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:30.918409 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.919231 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.919238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.919240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.919370 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:30.919714 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:30.919718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:30.919721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:30.920107 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:30.920329 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:30.920607 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 06:51:30.920812 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":354,"threads_started":1}
I20260619 06:51:30.922448 27070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:31.008333 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26947
I20260619 06:51:31.024921 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27081
Jun 19 06:51:31 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:51:31.040949 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:31.025835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.18.39.66:39443)
0619 06:51:31.026401 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:31.026408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:31.026434 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:51:31.026512 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:31.026518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:31.026728 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:31.026852 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:31.028042 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:31.029026 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:31.030335 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:31.030926 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:31.031031 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:31.035381 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:31.035400 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:31.035417 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:31.035464 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:31.038340 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:31.039017 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:31.039025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:31.039027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:31.039157 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:31.039459 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:31.039462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:31.039464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:31.039795 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:31.040047 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:31.040302 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 06:51:31.040507 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":288,"threads_started":1}
I20260619 06:51:31.042322 27204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:31.125965 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27081
I20260619 06:51:31.141710 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27215
Jun 19 06:51:31 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:51:31.158473 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:31.142673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46046 (local address 127.18.39.67:38227)
0619 06:51:31.143267 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:31.143275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:31.143304 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:51:31.143363 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:31.143369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:31.143585 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:31.143708 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:31.145043 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:31.146177 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:31.147672 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:31.148246 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:31.148345 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:31.152763 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:31.152775 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:31.152792 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:31.152831 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:31.155772 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:31.156433 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:31.156440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:31.156442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:31.156562 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:31.156900 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:31.156903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:31.156905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:31.157241 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:31.157581 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:31.157863 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 06:51:31.158082 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":295,"threads_started":1}
I20260619 06:51:31.159781 27338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:31.243186 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27215
I20260619 06:51:31.258754 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 26854
I20260619 06:51:31.259982 26916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:31.362922 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 26854
I20260619 06:51:31.384028 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38801
--webserver_interface=127.18.39.126
--webserver_port=44253
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:38801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:31.712755 27361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:31.713074 27361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:31.713236 27361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:31.720602 27361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:31.720724 27361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:31.720813 27361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:31.720924 27361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:31.721004 27361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:31.731245 27361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:38801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=44253
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:31.732703 27361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:31.734112 27361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:31.746207 27367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:31.746210 27366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:31.746593 27369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:31.747121 27361 server_base.cc:1034] running on GCE node
I20260619 06:51:31.747905 27361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:31.749416 27361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:31.750633 27361 hybrid_clock.cc:648] HybridClock initialized: now 1781851891750565 us; error 68 us; skew 500 ppm
I20260619 06:51:31.750993 27361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:31 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851891, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:31.754158 27361 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:51:31.755774 27361 webserver.cc:460] Webserver started at http://127.18.39.126:44253/ using document root <none> and password file <none>
I20260619 06:51:31.756500 27361 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:31.756660 27361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:31.762367 27361 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:51:31.766028 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:31.767711 27361 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:51:31.767923 27361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "678e454d43004459ae976bb60a20de94"
format_stamp: "Formatted at 2026-06-19 06:51:28 on dist-test-slave-1h5s"
server_key: "1b73c19484d9985bdc747f79efc3bc62"
server_key_iv: "72c0e16370ec86af9e89dc3c4e3b6097"
server_key_version: "encryptionkey@0"
I20260619 06:51:31.768963 27361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:31.785692 27361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:31.790073 27361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:31.790357 27361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:31.809093 27361 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:38801
I20260619 06:51:31.809114 27427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:38801 every 8 connection(s)
I20260619 06:51:31.810710 27361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:31.816570 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27361
I20260619 06:51:31.817595 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:46517
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:38801
--webserver_port=42183
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:51:31.818418 27428 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:51:31.823875 27428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Bootstrap starting.
I20260619 06:51:31.843356 27428 log.cc:824] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:31.852231 27428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:51:31.852784 27428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Bootstrap complete.
I20260619 06:51:31.861776 27428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:31.862574 27428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 678e454d43004459ae976bb60a20de94, State: Initialized, Role: FOLLOWER
I20260619 06:51:31.863198 27428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:31.863440 27428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:31.863575 27428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:31.863759 27428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:51:31.867518 27428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:31.868016 27428 leader_election.cc:302] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 678e454d43004459ae976bb60a20de94; no voters:
I20260619 06:51:31.868649 27428 leader_election.cc:288] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:51:31.868789 27432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:51:31.869910 27432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [term 2 LEADER]: Becoming Leader. State: Replica: 678e454d43004459ae976bb60a20de94, State: Running, Role: LEADER
I20260619 06:51:31.870602 27432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } }
I20260619 06:51:31.870959 27428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:31.873049 27434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 678e454d43004459ae976bb60a20de94. Latest consensus state: current_term: 2 leader_uuid: "678e454d43004459ae976bb60a20de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } } }
I20260619 06:51:31.873416 27434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:31.873867 27433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "678e454d43004459ae976bb60a20de94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "678e454d43004459ae976bb60a20de94" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38801 } } }
I20260619 06:51:31.874151 27433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:31.884824 27437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:31.891811 27437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:31.893280 27437 catalog_manager.cc:1259] Loaded cluster ID: 75783a99fec842e6b6ce04b148c6c1c1
I20260619 06:51:31.893476 27437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:31.897709 27437 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:31.899187 27437 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 678e454d43004459ae976bb60a20de94: Loaded TSK: 0
I20260619 06:51:31.900767 27437 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:51:32.166437 27430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:32.166731 27430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:32.166872 27430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:32.174741 27430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:32.174860 27430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:32.175025 27430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:32.185313 27430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.65.keytab
--principal=oryx/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:46517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=42183
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:32.186754 27430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:32.188068 27430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:32.202540 27455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:32.202710 27454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:32.202569 27457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:32.203406 27430 server_base.cc:1034] running on GCE node
I20260619 06:51:32.204041 27430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:32.205003 27430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:32.206261 27430 hybrid_clock.cc:648] HybridClock initialized: now 1781851892206199 us; error 50 us; skew 500 ppm
I20260619 06:51:32.206665 27430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851892, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:32.209975 27430 init.cc:377] Logged in from keytab as oryx/127.18.39.65@KRBTEST.COM (short username oryx)
I20260619 06:51:32.211704 27430 webserver.cc:460] Webserver started at http://127.18.39.65:42183/ using document root <none> and password file <none>
I20260619 06:51:32.212419 27430 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:32.212572 27430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:32.218345 27430 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:51:32.221635 27464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.223268 27430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260619 06:51:32.223445 27430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "b8d30b76e3a14d74bd528992a13c1544"
format_stamp: "Formatted at 2026-06-19 06:51:29 on dist-test-slave-1h5s"
server_key: "b93e24d6c1c04ea6d68a8c6b28d55d66"
server_key_iv: "066c55e7cd69be2f88a25e9fc39fc9be"
server_key_version: "encryptionkey@0"
I20260619 06:51:32.224402 27430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:32.251816 27430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:32.256376 27430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:32.256672 27430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:32.257786 27430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:32.259122 27430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:32.259212 27430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.259367 27430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:32.259461 27430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.292764 27430 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:46517
I20260619 06:51:32.292867 27576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:46517 every 8 connection(s)
I20260619 06:51:32.294389 27430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:32.302517 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27430
I20260619 06:51:32.303141 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:39443
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:38801
--webserver_port=43679
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:32.636004 27581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:32.636283 27581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:32.636441 27581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:32.644254 27581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:32.644375 27581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:32.644586 27581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:32.654428 27581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.66.keytab
--principal=oryx/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:39443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=43679
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:32.655761 27581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:32.657023 27581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:32.675909 27588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:32.675894 27587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:32.678678 27590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:32.679811 27581 server_base.cc:1034] running on GCE node
I20260619 06:51:32.680510 27581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:32.681631 27581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:32.683096 27581 hybrid_clock.cc:648] HybridClock initialized: now 1781851892683018 us; error 80 us; skew 500 ppm
I20260619 06:51:32.683458 27581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851892, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:32.687285 27581 init.cc:377] Logged in from keytab as oryx/127.18.39.66@KRBTEST.COM (short username oryx)
I20260619 06:51:32.689011 27581 webserver.cc:460] Webserver started at http://127.18.39.66:43679/ using document root <none> and password file <none>
I20260619 06:51:32.689797 27581 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:32.689982 27581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:32.695567 27581 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 06:51:32.698408 27597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.699841 27581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:51:32.700022 27581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "3f3468c38a8442c4b81c17f291e80793"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "9be53f855525126398bb420dd9cf1777"
server_key_iv: "f4b0d2cc4bb7d6f98b09d9fee299432c"
server_key_version: "encryptionkey@0"
I20260619 06:51:32.700945 27581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:32.726917 27581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:32.731550 27581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:32.731839 27581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:32.732913 27581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:32.734421 27581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:32.734514 27581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.734649 27581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:32.734719 27581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:32.785404 27581 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:39443
I20260619 06:51:32.785568 27709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:39443 every 8 connection(s)
I20260619 06:51:32.787016 27581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:32.796439 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27581
I20260619 06:51:32.797165 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:38227
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:38801
--webserver_port=40379
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:32 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:33.395710 27714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:33.396056 27714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:33.396229 27714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:33.403972 27714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:33.404117 27714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:33.404305 27714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:33.414736 27714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/krb5kdc/oryx_127.18.39.67.keytab
--principal=oryx/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:38227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=40379
--tserver_master_addrs=127.18.39.126:38801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:33.416240 27714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:33.417629 27714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:33.434625 27721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:33.434674 27720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:33.442943 27723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:33.444089 27714 server_base.cc:1034] running on GCE node
I20260619 06:51:33.445273 27714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:33.447088 27714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:33.448410 27714 hybrid_clock.cc:648] HybridClock initialized: now 1781851893448333 us; error 44 us; skew 500 ppm
I20260619 06:51:33.448990 27714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851893, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:33.454809 27714 init.cc:377] Logged in from keytab as oryx/127.18.39.67@KRBTEST.COM (short username oryx)
I20260619 06:51:33.457283 27714 webserver.cc:460] Webserver started at http://127.18.39.67:40379/ using document root <none> and password file <none>
I20260619 06:51:33.458359 27714 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:33.458516 27714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:33.466876 27714 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 06:51:33.470760 27730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:33.473063 27714 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:51:33.473309 27714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "28acff3d075443939ce38ea5afe2103c"
format_stamp: "Formatted at 2026-06-19 06:51:30 on dist-test-slave-1h5s"
server_key: "b537bfbbfff0867aeaa53b92bcede4d5"
server_key_iv: "17f28465b62b7d18afa7889b36d008b7"
server_key_version: "encryptionkey@0"
I20260619 06:51:33.474721 27714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:33.504782 27714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
W20260619 06:51:33.506204 27580 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.198s user 0.006s sys 0.008s
W20260619 06:51:33.508365 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:32.297242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:59505 (local address 127.18.39.126:38801)
0619 06:51:32.297961 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:32.297972 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:32.298382 (+ 410us) server_negotiation.cc:407] Connection header received
0619 06:51:32.300481 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:32.300494 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:32.300885 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:32.301180 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:32.303007 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.304292 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.305914 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.306573 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.306780 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.507683 (+1200903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:59505: BlockingRecv error: recv got EOF from 127.18.39.65:59505 (error 108)
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":251,"threads_started":1}
I20260619 06:51:33.508905 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:32.297305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:59505)
0619 06:51:32.298194 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:32.298246 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:32.299960 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:32.301430 (+ 1470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:32.301438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:32.302152 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:32.302808 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:32.302829 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.304462 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.304468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:32.305768 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:32.305781 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.306817 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.306825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:32.307094 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:32.307802 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:32.307826 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:33.507427 (+1199601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":339,"threads_started":1}
W20260619 06:51:33.509910 27577 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:38801 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.39.126:38801: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
I20260619 06:51:33.512009 27714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:33.512310 27714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:33.513411 27714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:33.515280 27714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:33.515390 27714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:33.515502 27714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:33.515575 27714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:33.525718 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:33.511114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:41869 (local address 127.18.39.126:38801)
0619 06:51:33.511289 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:33.511295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:33.511633 (+ 338us) server_negotiation.cc:407] Connection header received
0619 06:51:33.511856 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:33.511861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:33.511942 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:33.512061 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:33.513485 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.514643 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.516720 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.517395 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.517534 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.525303 (+ 7769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:41869: BlockingRecv error: recv got EOF from 127.18.39.65:41869 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:51:33.526129 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:33.511095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:41869)
0619 06:51:33.511504 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:33.511532 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:33.511706 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:33.512234 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:33.512242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:33.512872 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:33.513202 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.513219 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.514935 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.514940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.516501 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.516518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.517636 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.517647 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.517940 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.518809 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:33.518845 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:33.525628 (+ 6783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226}
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:33.542361 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:33.527462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:57665)
0619 06:51:33.527828 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:33.527856 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:33.528044 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:33.528701 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:33.528708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:33.529289 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:33.529632 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.529645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.531678 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.531686 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.533764 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.533783 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.535227 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.535235 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.535504 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.536345 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:33.536376 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:33.541967 (+ 5591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198}
W20260619 06:51:33.542941 27577 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:51:33.543318 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:33.527683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:57665 (local address 127.18.39.126:38801)
0619 06:51:33.527896 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:33.527903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:33.527938 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:51:33.528217 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:33.528223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:33.528334 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:33.528469 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:33.529851 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.531381 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.533973 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.534786 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.534906 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.543025 (+ 8119us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:57665: BlockingRecv error: recv got EOF from 127.18.39.65:57665 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:51:33.575130 27714 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:38227
I20260619 06:51:33.575173 27842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:38227 every 8 connection(s)
I20260619 06:51:33.576982 27714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:33.586983 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27714
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:33 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:33.624531 27847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:33.588032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.18.39.126:38801)
0619 06:51:33.597637 (+ 9605us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:33.597646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:33.598306 (+ 660us) server_negotiation.cc:407] Connection header received
0619 06:51:33.598530 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:33.598538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:33.598658 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:33.598764 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:33.602984 (+ 4220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.604812 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.607155 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.608161 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.608350 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.612605 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:33.612623 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:33.612662 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:33.612748 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:33.618773 (+ 6025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:33.620231 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:33.620246 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:33.620262 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:33.620594 (+ 332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:33.621205 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:33.621213 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:33.621218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:33.622214 (+ 996us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:33.622687 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:33.623302 (+ 615us) server_negotiation.cc:299] Negotiation successful
0619 06:51:33.623900 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9398,"thread_start_us":7630,"threads_started":1}
W20260619 06:51:34.082149 27713 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.279s user 0.007s sys 0.011s
I20260619 06:51:34.083775 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:32.791240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:34611)
0619 06:51:32.792165 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:32.792206 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:32.794065 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:32.794823 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:32.794832 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:32.795355 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:32.796102 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:32.796129 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.797683 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.797691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:32.799967 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:32.799985 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.801385 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.801395 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:32.801725 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:32.802704 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:32.802738 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.082893 (+1280155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":391,"threads_started":1}
W20260619 06:51:34.083822 27712 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:32.789975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:34611 (local address 127.18.39.126:38801)
0619 06:51:32.790596 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:32.790600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:32.792334 (+ 1734us) server_negotiation.cc:407] Connection header received
0619 06:51:32.794344 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:32.794350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:32.794457 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:32.794577 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:32.796338 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.797489 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.800166 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:32.800888 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:32.801024 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.083329 (+1282305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:34611: BlockingRecv error: recv got EOF from 127.18.39.66:34611 (error 108)
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":417,"threads_started":1}
W20260619 06:51:34.084661 27710 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:38801 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.39.126:38801: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:34.097980 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:34.085380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:47399)
0619 06:51:34.085565 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:34.085586 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:34.085698 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:34.086107 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:34.086111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:34.086504 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:34.086659 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.086667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.087684 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.087689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.088647 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.088658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.089391 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.089398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.089554 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.094096 (+ 4542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:34.094115 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.097674 (+ 3559us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 06:51:34.098106 27712 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:34.085654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47399 (local address 127.18.39.126:38801)
0619 06:51:34.085811 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:34.085816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:34.085832 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:51:34.085882 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:34.085887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:34.085962 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:34.086055 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:34.086802 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.087542 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.088789 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.089186 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.089323 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.097866 (+ 8543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:47399: BlockingRecv error: recv got EOF from 127.18.39.66:47399 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:34.110981 27712 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:34.099213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:57375 (local address 127.18.39.126:38801)
0619 06:51:34.099459 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:34.099470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:34.099724 (+ 254us) server_negotiation.cc:407] Connection header received
0619 06:51:34.099873 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:34.099882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:34.099990 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:34.100134 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:34.100939 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.102682 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.103978 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.105034 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.105216 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.110538 (+ 5322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:57375: BlockingRecv error: recv got EOF from 127.18.39.66:57375 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:51:34.111786 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:34.098994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:57375)
0619 06:51:34.099630 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:34.099645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:34.099745 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:34.100137 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:34.100140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:34.100568 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:34.100796 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.100803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.102848 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.102852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.103798 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.103809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.105223 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.105229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.105395 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.106113 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:34.106130 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.111281 (+ 5151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":281,"threads_started":1}
W20260619 06:51:34.112232 27710 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:51:34.557698 27712 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:34.543732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42449 (local address 127.18.39.126:38801)
0619 06:51:34.544014 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:34.544024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:34.544272 (+ 248us) server_negotiation.cc:407] Connection header received
0619 06:51:34.544435 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:34.544444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:34.544604 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:34.544801 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:34.545611 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.547229 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.550140 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.550552 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.550652 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.557530 (+ 6878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:42449: BlockingRecv error: recv got EOF from 127.18.39.65:42449 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 06:51:34.557837 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:34.543583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:42449)
0619 06:51:34.544162 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:34.544179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:34.544296 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:34.544808 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:34.544811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:34.545214 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:34.545453 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.545464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.547515 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.547526 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.549922 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.549947 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.550791 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.550801 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.551087 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.551908 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:34.551944 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.557287 (+ 5343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":341,"threads_started":1}
W20260619 06:51:34.718318 27845 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.122s user 0.002s sys 0.015s
W20260619 06:51:34.719484 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:33.580660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60267 (local address 127.18.39.126:38801)
0619 06:51:33.580905 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:33.580911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:33.582206 (+ 1295us) server_negotiation.cc:407] Connection header received
0619 06:51:33.584260 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:33.584270 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:33.584398 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:33.584574 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:33.586517 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.588224 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.592031 (+ 3807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.593436 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.593606 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.719269 (+1125663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:60267: BlockingRecv error: recv got EOF from 127.18.39.67:60267 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 06:51:34.720001 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:33.581144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:60267)
0619 06:51:33.582005 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:33.582061 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:33.583941 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:33.584795 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:33.584812 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:33.585575 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:33.586284 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.586303 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.588545 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.588554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.591691 (+ 3137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:33.591723 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:33.594191 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:33.594201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:33.594712 (+ 511us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:33.595833 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:33.595901 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.719090 (+1123189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":379,"threads_started":1}
W20260619 06:51:34.721017 27843 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:38801 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.39.126:38801: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:34.730367 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:34.721772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:39889)
0619 06:51:34.721978 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:34.721998 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:34.722122 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:34.722441 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:34.722446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:34.722814 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:34.722971 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.722979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.724049 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.724055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.725018 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.725029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.725883 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.725889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.726067 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.726622 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:34.726638 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.730052 (+ 3414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 06:51:34.730486 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:34.721833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:39889 (local address 127.18.39.126:38801)
0619 06:51:34.722060 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:34.722067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:34.722084 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:51:34.722244 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:34.722249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:34.722316 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:34.722415 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:34.723136 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.723904 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.725179 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.725703 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.725835 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.730247 (+ 4412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:39889: BlockingRecv error: recv got EOF from 127.18.39.67:39889 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:34 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:34.738847 27845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:34.731075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:33831)
0619 06:51:34.731233 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:34.731251 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:34.731351 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:34.731650 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:34.731653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:34.731923 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:34.732082 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.732089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.733031 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.733036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.734370 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:34.734384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.735038 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.735042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:34.735164 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.735685 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:34.735701 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:34.738652 (+ 2951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 06:51:34.739104 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:34.731171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:33831 (local address 127.18.39.126:38801)
0619 06:51:34.731328 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:34.731332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:34.731348 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:51:34.731442 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:34.731446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:34.731502 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:34.731609 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:34.732189 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.732902 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.734509 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:34.734919 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:34.735011 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:34.738855 (+ 3844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:33831: BlockingRecv error: recv got EOF from 127.18.39.67:33831 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 06:51:34.739174 27843 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:35.121224 27851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:35.112897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:60157)
0619 06:51:35.113486 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:35.113502 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:35.113614 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:35.113872 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:35.113876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:35.114215 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:35.114362 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.114370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.115295 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.115299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.116231 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.116242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.116863 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.116867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.116992 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.117670 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:35.117688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:35.121009 (+ 3321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":287,"threads_started":1}
W20260619 06:51:35.121495 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:35.112954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:60157 (local address 127.18.39.126:38801)
0619 06:51:35.113217 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:35.113225 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:35.113584 (+ 359us) server_negotiation.cc:407] Connection header received
0619 06:51:35.113693 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:35.113696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:35.113757 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:35.113832 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:35.114498 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.115182 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.116342 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.116737 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.116839 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.121199 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:60157: BlockingRecv error: recv got EOF from 127.18.39.66:60157 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:35.568060 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:35.558947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:47459)
0619 06:51:35.559510 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:35.559526 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:35.559701 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:35.560038 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:35.560042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:35.560509 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:35.560686 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.560695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.561740 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.561747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.562655 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.562666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.563418 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.563424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.563605 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.564214 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:35.564232 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:35.567813 (+ 3581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":302,"threads_started":1}
W20260619 06:51:35.568257 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:35.559043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:47459 (local address 127.18.39.126:38801)
0619 06:51:35.559312 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:35.559320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:35.559621 (+ 301us) server_negotiation.cc:407] Connection header received
0619 06:51:35.559834 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:35.559843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:35.559905 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:35.560007 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:35.560833 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.561609 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.562797 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.563259 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.563406 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.568005 (+ 4599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:47459: BlockingRecv error: recv got EOF from 127.18.39.65:47459 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:35 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:35.749274 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:35.739816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:45997)
0619 06:51:35.740364 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:35.740383 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:35.740560 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:35.740914 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:35.740919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:35.741443 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:35.741592 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.741602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.742642 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.742649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.743609 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:35.743620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.744366 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.744373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:35.744547 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.745140 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:35.745158 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:35.749034 (+ 3876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":311,"threads_started":1}
W20260619 06:51:35.749464 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:35.739890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:45997 (local address 127.18.39.126:38801)
0619 06:51:35.740118 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:35.740134 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:35.740489 (+ 355us) server_negotiation.cc:407] Connection header received
0619 06:51:35.740686 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:35.740697 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:35.740757 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:35.740862 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:35.741739 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.742494 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.743753 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:35.744161 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:35.744317 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:35.749210 (+ 4893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:45997: BlockingRecv error: recv got EOF from 127.18.39.67:45997 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:36.131511 27854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:36.122221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:38889)
0619 06:51:36.122737 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:36.122753 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:36.122869 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:36.123169 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:36.123172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:36.123531 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:36.123702 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.123709 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.124741 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.124745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.125738 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.125750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.126637 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.126643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.126833 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.127614 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:36.127639 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:36.131246 (+ 3607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":303,"threads_started":1}
W20260619 06:51:36.131788 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:36.122371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:38889 (local address 127.18.39.126:38801)
0619 06:51:36.122554 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:36.122559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:36.122828 (+ 269us) server_negotiation.cc:407] Connection header received
0619 06:51:36.122962 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:36.122966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:36.123027 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:36.123114 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:36.123873 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.124609 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.125890 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.126460 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.126582 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.131511 (+ 4929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:38889: BlockingRecv error: recv got EOF from 127.18.39.66:38889 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:36.578208 27855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:36.569102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:60235)
0619 06:51:36.569709 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:36.569725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:36.569835 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:36.570111 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:36.570114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:36.570482 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:36.570637 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.570644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.571748 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.571752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.572726 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.572738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.573509 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.573513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.573625 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.574205 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:36.574222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:36.577983 (+ 3761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":351,"threads_started":1}
W20260619 06:51:36.578446 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:36.569252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:60235 (local address 127.18.39.126:38801)
0619 06:51:36.569512 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:36.569518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:36.569797 (+ 279us) server_negotiation.cc:407] Connection header received
0619 06:51:36.569928 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:36.569932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:36.569991 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:36.570070 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:36.570757 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.571617 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.572860 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.573393 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.573523 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.578176 (+ 4653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:60235: BlockingRecv error: recv got EOF from 127.18.39.65:60235 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:36 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:36.759624 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:36.750317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:37411)
0619 06:51:36.750841 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:36.750856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:36.751034 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:36.751388 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:36.751392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:36.751952 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:36.752211 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.752224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.753277 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.753284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.754279 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:36.754289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.755000 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.755006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:36.755193 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.755770 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:36.755787 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:36.759233 (+ 3446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":296,"threads_started":1}
W20260619 06:51:36.759656 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:36.750398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:37411 (local address 127.18.39.126:38801)
0619 06:51:36.750689 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:36.750695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:36.750952 (+ 257us) server_negotiation.cc:407] Connection header received
0619 06:51:36.751179 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:36.751184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:36.751243 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:36.751363 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:36.752367 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.753124 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.754423 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:36.754845 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:36.754980 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:36.759438 (+ 4458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:37411: BlockingRecv error: recv got EOF from 127.18.39.67:37411 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:37.141563 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:37.132751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:35425)
0619 06:51:37.133461 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:37.133478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:37.133590 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:37.133965 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:37.133968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:37.134332 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:37.134509 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.134517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.135477 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.135480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.136427 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.136439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.137126 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.137130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.137259 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.137911 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:37.137928 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:37.141292 (+ 3364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":417,"threads_started":1}
W20260619 06:51:37.141831 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:37.132811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:35425 (local address 127.18.39.126:38801)
0619 06:51:37.133051 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:37.133060 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:37.133560 (+ 500us) server_negotiation.cc:407] Connection header received
0619 06:51:37.133722 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:37.133728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:37.133820 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:37.133916 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:37.134640 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.135358 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.136572 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.136996 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.137101 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.141581 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:35425: BlockingRecv error: recv got EOF from 127.18.39.66:35425 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:37.588198 27858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:37.579111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:35047)
0619 06:51:37.579646 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:37.579663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:37.579768 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:37.580145 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:37.580149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:37.580506 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:37.580668 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.580676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.581732 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.581736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.582719 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.582729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.583365 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.583370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.583489 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.584041 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:37.584058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:37.587954 (+ 3896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
W20260619 06:51:37.588341 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:37.579233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:35047 (local address 127.18.39.126:38801)
0619 06:51:37.579393 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:37.579397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:37.579732 (+ 335us) server_negotiation.cc:407] Connection header received
0619 06:51:37.579940 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:37.579943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:37.580000 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:37.580110 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:37.580814 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.581607 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.582847 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.583244 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.583335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.588146 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:35047: BlockingRecv error: recv got EOF from 127.18.39.65:35047 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:37 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:37.769564 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:37.760638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:33517)
0619 06:51:37.761224 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:37.761238 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:37.761416 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:37.761746 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:37.761751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:37.762114 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:37.762280 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.762288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.763344 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.763351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.764260 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:37.764270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.764947 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.764953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:37.765122 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.765773 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:37.765789 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:37.769256 (+ 3467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":350,"threads_started":1}
W20260619 06:51:37.769805 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:37.760708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:33517 (local address 127.18.39.126:38801)
0619 06:51:37.760903 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:37.760911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:37.761321 (+ 410us) server_negotiation.cc:407] Connection header received
0619 06:51:37.761553 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:37.761558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:37.761621 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:37.761719 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:37.762428 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.763224 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.764396 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:37.764804 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:37.764932 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:37.769562 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:33517: BlockingRecv error: recv got EOF from 127.18.39.67:33517 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:38.151599 27860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:38.142540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:52971)
0619 06:51:38.143074 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:38.143090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:38.143200 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:38.143500 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:38.143504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:38.143876 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:38.144034 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.144042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.145067 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.145071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.146175 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.146187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.146812 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.146818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.146940 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.147540 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:38.147558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:38.151251 (+ 3693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":295,"threads_started":1}
W20260619 06:51:38.151798 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:38.142621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:52971 (local address 127.18.39.126:38801)
0619 06:51:38.142782 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:38.142786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:38.143162 (+ 376us) server_negotiation.cc:407] Connection header received
0619 06:51:38.143294 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:38.143297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:38.143357 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:38.143437 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:38.144152 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.144927 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.146300 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.146687 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.146778 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.151509 (+ 4731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:52971: BlockingRecv error: recv got EOF from 127.18.39.66:52971 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:38.598654 27861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:38.589261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:42447)
0619 06:51:38.589829 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:38.589846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:38.590021 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:38.590350 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:38.590355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:38.590776 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:38.590938 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.590946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.591992 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.591999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.593083 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.593095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.593953 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.593959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.594143 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.594887 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:38.594905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:38.598424 (+ 3519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":325,"threads_started":1}
W20260619 06:51:38.598948 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:38.589432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:42447 (local address 127.18.39.126:38801)
0619 06:51:38.589669 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:38.589676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:38.589944 (+ 268us) server_negotiation.cc:407] Connection header received
0619 06:51:38.590148 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:38.590158 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:38.590220 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:38.590324 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:38.591081 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.591860 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.593226 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.593787 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.593923 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.598659 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:42447: BlockingRecv error: recv got EOF from 127.18.39.65:42447 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:38 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:38.780021 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:38.770520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:46087)
0619 06:51:38.771044 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:38.771061 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:38.771216 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:38.771620 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:38.771625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:38.772058 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:38.772268 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.772278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.773298 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.773304 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.774351 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:38.774363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.775048 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.775054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:38.775234 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.775885 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:38.775903 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:38.779606 (+ 3703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":266,"threads_started":1}
W20260619 06:51:38.780045 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:38.770593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:46087 (local address 127.18.39.126:38801)
0619 06:51:38.770825 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:38.770833 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:38.771163 (+ 330us) server_negotiation.cc:407] Connection header received
0619 06:51:38.771394 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:38.771398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:38.771475 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:38.771581 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:38.772412 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.773150 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.774493 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:38.774901 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:38.775034 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:38.779821 (+ 4787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:46087: BlockingRecv error: recv got EOF from 127.18.39.67:46087 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:39.166043 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:39.152599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:55519)
0619 06:51:39.153128 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:39.153145 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:39.153259 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:39.153612 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:39.153616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:39.153991 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:39.154141 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.154148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.155186 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.155190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.156110 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.156120 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.156945 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.156951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.157097 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.162310 (+ 5213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:39.162331 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:39.165807 (+ 3476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":294,"threads_started":1}
W20260619 06:51:39.166318 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:39.152830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:55519 (local address 127.18.39.126:38801)
0619 06:51:39.153013 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:39.153017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:39.153273 (+ 256us) server_negotiation.cc:407] Connection header received
0619 06:51:39.153388 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:39.153392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:39.153458 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:39.153545 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:39.154288 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.155069 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.156243 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.156695 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.156793 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.166033 (+ 9240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:55519: BlockingRecv error: recv got EOF from 127.18.39.66:55519 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:39.610009 27864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:39.600000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:53813)
0619 06:51:39.600837 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:39.600865 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:39.600980 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:39.601611 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:39.601615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:39.602068 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:39.602249 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.602258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.603319 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.603324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.604539 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.604552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.605411 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.605417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.605628 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.606344 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:39.606362 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:39.609773 (+ 3411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":621,"threads_started":1}
W20260619 06:51:39.610258 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:39.600105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:53813 (local address 127.18.39.126:38801)
0619 06:51:39.600321 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:39.600325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:39.600962 (+ 637us) server_negotiation.cc:407] Connection header received
0619 06:51:39.601077 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:39.601084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:39.601175 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:39.601311 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:39.602456 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.603154 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.604688 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.605161 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.605351 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.609985 (+ 4634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:53813: BlockingRecv error: recv got EOF from 127.18.39.65:53813 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:39 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:39.790380 27865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:39.781046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:60641)
0619 06:51:39.781591 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:39.781607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:39.781721 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:39.782022 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:39.782025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:39.782414 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:39.782562 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.782571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.783592 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.783597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.785032 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:39.785047 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.786050 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.786055 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:39.786179 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.786809 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:39.786826 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:39.790164 (+ 3338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":309,"threads_started":1}
W20260619 06:51:39.790580 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:39.781139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60641 (local address 127.18.39.126:38801)
0619 06:51:39.781437 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:39.781444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:39.781686 (+ 242us) server_negotiation.cc:407] Connection header received
0619 06:51:39.781814 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:39.781818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:39.781900 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:39.781982 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:39.782677 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.783418 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.785179 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:39.785901 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:39.786014 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:39.790346 (+ 4332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:60641: BlockingRecv error: recv got EOF from 127.18.39.67:60641 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:40.176460 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:40.166941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:41815)
0619 06:51:40.167414 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:40.167429 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:40.167544 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:40.167830 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:40.167833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:40.168237 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:40.168409 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.168418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.169676 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.169680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.170608 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.170619 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.171360 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.171365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.171554 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.172268 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:40.172296 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:40.176216 (+ 3920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":272,"threads_started":1}
W20260619 06:51:40.176759 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:40.167003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:41815 (local address 127.18.39.126:38801)
0619 06:51:40.167263 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:40.167270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:40.167506 (+ 236us) server_negotiation.cc:407] Connection header received
0619 06:51:40.167639 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:40.167643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:40.167705 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:40.167787 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:40.168589 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.169544 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.170763 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.171198 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.171303 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.176511 (+ 5208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:41815: BlockingRecv error: recv got EOF from 127.18.39.66:41815 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:40.621771 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:40.611058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:45207)
0619 06:51:40.611715 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:40.611733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:40.611916 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:40.612299 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:40.612304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:40.612832 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:40.613036 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.613046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.614222 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.614230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.615492 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.615505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.616265 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.616270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.616447 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.617151 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:40.617173 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:40.621480 (+ 4307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":292,"threads_started":1}
W20260619 06:51:40.622009 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:40.611160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:45207 (local address 127.18.39.126:38801)
0619 06:51:40.611422 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:40.611431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:40.611834 (+ 403us) server_negotiation.cc:407] Connection header received
0619 06:51:40.612051 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:40.612057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:40.612143 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:40.612258 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:40.613198 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.614071 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.615636 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.616093 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.616231 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.621740 (+ 5509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:45207: BlockingRecv error: recv got EOF from 127.18.39.65:45207 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:40 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:40.802647 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:40.791516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:48149)
0619 06:51:40.792116 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:40.792167 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:40.792310 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:40.792692 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:40.792696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:40.793167 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:40.793408 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.793419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.794511 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.794515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.795509 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:40.795521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.796259 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.796264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:40.796395 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.797112 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:40.797138 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:40.802220 (+ 5082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":349,"threads_started":1}
W20260619 06:51:40.802752 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:40.791696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:48149 (local address 127.18.39.126:38801)
0619 06:51:40.791998 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:40.792006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:40.792268 (+ 262us) server_negotiation.cc:407] Connection header received
0619 06:51:40.792432 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:40.792437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:40.792514 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:40.792648 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:40.793558 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.794379 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.795663 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:40.796105 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:40.796217 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:40.802456 (+ 6239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:48149: BlockingRecv error: recv got EOF from 127.18.39.67:48149 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:41.188022 27869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:41.177730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:33563)
0619 06:51:41.178434 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:41.178453 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:41.178593 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:41.179014 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:41.179017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:41.179563 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:41.179767 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.179777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.180905 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.180909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.181985 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.181995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.182698 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.182703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.182829 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.183587 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:41.183611 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:41.187716 (+ 4105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":351,"threads_started":1}
W20260619 06:51:41.188241 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:41.177840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:33563 (local address 127.18.39.126:38801)
0619 06:51:41.178111 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:41.178117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:41.178536 (+ 419us) server_negotiation.cc:407] Connection header received
0619 06:51:41.178776 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:41.178781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:41.178870 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:41.178963 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:41.179959 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.180764 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.182103 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.182541 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.182652 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.187938 (+ 5286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:33563: BlockingRecv error: recv got EOF from 127.18.39.66:33563 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:41.633216 27870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:41.622900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:55479)
0619 06:51:41.623536 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:41.623554 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:41.623681 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:41.624049 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:41.624053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:41.624521 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:41.624701 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.624710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.626043 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.626048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.627050 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.627062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.628132 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.628139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.628344 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.629059 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:41.629096 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:41.632965 (+ 3869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":324,"threads_started":1}
W20260619 06:51:41.633630 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:41.623055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:55479 (local address 127.18.39.126:38801)
0619 06:51:41.623264 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:41.623269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:41.623642 (+ 373us) server_negotiation.cc:407] Connection header received
0619 06:51:41.623795 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:41.623800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:41.623893 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:41.623995 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:41.624876 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.625890 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.627242 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.627878 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.628041 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.633283 (+ 5242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:55479: BlockingRecv error: recv got EOF from 127.18.39.65:55479 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:41 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:41.814227 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:41.803790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:33315)
0619 06:51:41.804620 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:41.804643 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:41.804788 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:41.805181 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:41.805187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:41.805788 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:41.806013 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.806023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.807067 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.807072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.808035 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:41.808045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.808781 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.808787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:41.808957 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.809659 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:41.809683 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:41.813879 (+ 4196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":480,"threads_started":1}
W20260619 06:51:41.814235 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:41.803940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:33315 (local address 127.18.39.126:38801)
0619 06:51:41.804131 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:41.804139 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:41.804731 (+ 592us) server_negotiation.cc:407] Connection header received
0619 06:51:41.804928 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:41.804934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:41.805038 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:41.805145 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:41.806165 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.806921 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.808187 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:41.808626 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:41.808765 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:41.814046 (+ 5281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:33315: BlockingRecv error: recv got EOF from 127.18.39.67:33315 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:42.198695 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:42.189095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:47951)
0619 06:51:42.189684 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:42.189701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:42.189865 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:42.190224 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:42.190227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:42.190820 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:42.191063 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.191075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.192523 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.192526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.193499 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.193510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.194190 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.194194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.194302 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.194929 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:42.194947 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:42.198446 (+ 3499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":343,"threads_started":1}
W20260619 06:51:42.198916 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:42.189185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47951 (local address 127.18.39.126:38801)
0619 06:51:42.189478 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:42.189485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:42.189786 (+ 301us) server_negotiation.cc:407] Connection header received
0619 06:51:42.189991 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:42.189995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:42.190056 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:42.190149 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:42.191216 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.192396 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.193632 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.194039 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.194161 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.198638 (+ 4477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:47951: BlockingRecv error: recv got EOF from 127.18.39.66:47951 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:42.643672 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:42.634293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:50703)
0619 06:51:42.634874 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:42.634891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:42.635006 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:42.635324 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:42.635327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:42.635719 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:42.635856 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.635864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.636937 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.636941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.637964 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.637977 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.638708 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.638713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.638838 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.639489 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:42.639509 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:42.643451 (+ 3942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":342,"threads_started":1}
W20260619 06:51:42.643867 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:42.634334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:50703 (local address 127.18.39.126:38801)
0619 06:51:42.634527 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:42.634532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:42.634974 (+ 442us) server_negotiation.cc:407] Connection header received
0619 06:51:42.635128 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:42.635132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:42.635195 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:42.635282 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:42.635984 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.636821 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.638114 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.638564 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.638665 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.643641 (+ 4976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:50703: BlockingRecv error: recv got EOF from 127.18.39.65:50703 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:42 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:42.824640 27874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:42.815195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:34537)
0619 06:51:42.815778 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:42.815793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:42.815956 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:42.816283 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:42.816288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:42.816792 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:42.816959 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.816968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.818084 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.818090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.819024 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:42.819034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.819831 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.819839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:42.820006 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.820683 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:42.820700 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:42.824302 (+ 3602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":322,"threads_started":1}
W20260619 06:51:42.824707 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:42.815283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34537 (local address 127.18.39.126:38801)
0619 06:51:42.815524 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:42.815532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:42.815888 (+ 356us) server_negotiation.cc:407] Connection header received
0619 06:51:42.816072 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:42.816079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:42.816157 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:42.816258 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:42.817106 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.817946 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.819164 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:42.819629 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:42.819757 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:42.824497 (+ 4740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:34537: BlockingRecv error: recv got EOF from 127.18.39.67:34537 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:43.209210 27875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:43.199808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:45085)
0619 06:51:43.200380 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:43.200396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:43.200556 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:43.200903 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:43.200907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:43.201390 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:43.201535 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.201543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.202529 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.202538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.203489 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.203499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.204220 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.204226 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.204411 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.205007 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:43.205023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:43.208984 (+ 3961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":284,"threads_started":1}
W20260619 06:51:43.209506 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:43.199903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:45085 (local address 127.18.39.126:38801)
0619 06:51:43.200160 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:43.200168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:43.200493 (+ 325us) server_negotiation.cc:407] Connection header received
0619 06:51:43.200684 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:43.200691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:43.200755 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:43.200879 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:43.201689 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.202396 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.203637 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.204052 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.204205 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.209185 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:45085: BlockingRecv error: recv got EOF from 127.18.39.66:45085 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:43.653589 27876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:43.644741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:45143)
0619 06:51:43.645253 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:43.645270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:43.645430 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:43.645737 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:43.645741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:43.646141 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:43.646298 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.646306 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.647252 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.647256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.648252 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.648263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.648918 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.648922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.649036 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.649702 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:43.649720 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:43.653201 (+ 3481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":294,"threads_started":1}
W20260619 06:51:43.653712 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:43.644815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:45143 (local address 127.18.39.126:38801)
0619 06:51:43.645083 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:43.645090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:43.645376 (+ 286us) server_negotiation.cc:407] Connection header received
0619 06:51:43.645537 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:43.645541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:43.645604 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:43.645686 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:43.646426 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.647134 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.648372 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.648765 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.648854 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.653411 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:45143: BlockingRecv error: recv got EOF from 127.18.39.65:45143 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:43 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:43.834785 27877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:43.825944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:45433)
0619 06:51:43.826520 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:43.826542 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:43.826667 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:43.826967 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:43.826971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:43.827440 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:43.827586 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.827595 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.828652 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.828656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.829785 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:43.829796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.830430 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.830434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:43.830553 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.831120 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:43.831136 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:43.834573 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":266,"threads_started":1}
W20260619 06:51:43.835070 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:43.825998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:45433 (local address 127.18.39.126:38801)
0619 06:51:43.826257 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:43.826263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:43.826634 (+ 371us) server_negotiation.cc:407] Connection header received
0619 06:51:43.826753 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:43.826756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:43.826836 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:43.826917 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:43.827738 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.828512 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.829919 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:43.830294 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:43.830397 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:43.834783 (+ 4386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:45433: BlockingRecv error: recv got EOF from 127.18.39.67:45433 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:44.223040 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:44.210303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:60861)
0619 06:51:44.210986 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:44.211002 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:44.211171 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:44.211545 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:44.211548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:44.211916 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:44.212088 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.212095 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.213163 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.213167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.214157 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.214169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.214839 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.214843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.214967 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.218926 (+ 3959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:44.218948 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:44.222688 (+ 3740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":397,"threads_started":1}
W20260619 06:51:44.223129 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:44.210380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:60861 (local address 127.18.39.126:38801)
0619 06:51:44.210599 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:44.210603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:44.211126 (+ 523us) server_negotiation.cc:407] Connection header received
0619 06:51:44.211346 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:44.211350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:44.211414 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:44.211493 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:44.212239 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.213028 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.214289 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.214701 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.214796 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.222935 (+ 8139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:60861: BlockingRecv error: recv got EOF from 127.18.39.66:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:44.663352 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:44.654603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:37225)
0619 06:51:44.655146 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:44.655162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:44.655270 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:44.655582 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:44.655585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:44.655986 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:44.656161 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.656169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.657155 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.657159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.658123 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.658135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.658820 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.658824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.658968 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.659598 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:44.659616 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:44.663022 (+ 3406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":315,"threads_started":1}
W20260619 06:51:44.663460 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:44.654739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:37225 (local address 127.18.39.126:38801)
0619 06:51:44.654960 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:44.654966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:44.655236 (+ 270us) server_negotiation.cc:407] Connection header received
0619 06:51:44.655368 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:44.655373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:44.655448 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:44.655524 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:44.656295 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.657043 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.658255 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.658680 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.658776 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.663212 (+ 4436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:37225: BlockingRecv error: recv got EOF from 127.18.39.65:37225 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:44 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:44.845324 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:44.835810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:53371)
0619 06:51:44.836399 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:44.836425 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:44.836541 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:44.836859 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:44.836862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:44.837229 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:44.837475 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.837486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.838766 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.838772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.839853 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:44.839865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.840573 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.840578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:44.840738 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.841382 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:44.841409 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:44.845071 (+ 3662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":359,"threads_started":1}
W20260619 06:51:44.845598 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:44.835913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:53371 (local address 127.18.39.126:38801)
0619 06:51:44.836086 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:44.836091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:44.836503 (+ 412us) server_negotiation.cc:407] Connection header received
0619 06:51:44.836644 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:44.836647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:44.836709 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:44.836800 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:44.837625 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.838629 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.839995 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:44.840420 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:44.840557 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:44.845278 (+ 4721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:53371: BlockingRecv error: recv got EOF from 127.18.39.67:53371 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:45.233428 27881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:45.224149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:58461)
0619 06:51:45.224707 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:45.224723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:45.224840 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:45.225148 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:45.225152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:45.225661 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:45.225843 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.225852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.226900 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.226905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.227845 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.227857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.228631 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.228636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.228751 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.229379 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:45.229404 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:45.233140 (+ 3736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":342,"threads_started":1}
W20260619 06:51:45.233623 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:45.224313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:58461 (local address 127.18.39.126:38801)
0619 06:51:45.224539 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:45.224548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:45.224823 (+ 275us) server_negotiation.cc:407] Connection header received
0619 06:51:45.224946 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:45.224950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:45.225011 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:45.225102 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:45.225996 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.226750 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.227978 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.228468 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.228579 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.233315 (+ 4736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:58461: BlockingRecv error: recv got EOF from 127.18.39.66:58461 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:45.673467 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:45.664306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:57489)
0619 06:51:45.664957 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:45.664974 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:45.665089 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:45.665519 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:45.665522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:45.665918 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:45.666097 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.666105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.667055 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.667058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.667962 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.667973 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.668641 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.668646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.668770 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.669413 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:45.669441 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:45.673091 (+ 3650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":401,"threads_started":1}
W20260619 06:51:45.673614 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:45.664382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:57489 (local address 127.18.39.126:38801)
0619 06:51:45.664626 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:45.664635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:45.665055 (+ 420us) server_negotiation.cc:407] Connection header received
0619 06:51:45.665221 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:45.665225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:45.665288 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:45.665476 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:45.666218 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.666937 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.668088 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.668488 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.668577 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.673297 (+ 4720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:57489: BlockingRecv error: recv got EOF from 127.18.39.65:57489 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:45 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:45.855677 27883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:45.846500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:43007)
0619 06:51:45.847018 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:45.847040 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:45.847206 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:45.847700 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:45.847705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:45.848241 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:45.848431 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.848440 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.849488 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.849496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.850415 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:45.850426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.851164 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.851170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:45.851357 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.851951 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:45.851984 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:45.855454 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":277,"threads_started":1}
W20260619 06:51:45.855960 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:45.846551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:43007 (local address 127.18.39.126:38801)
0619 06:51:45.846791 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:45.846799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:45.847220 (+ 421us) server_negotiation.cc:407] Connection header received
0619 06:51:45.847322 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:45.847327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:45.847533 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:45.847675 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:45.848580 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.849308 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.850557 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:45.851002 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:45.851128 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:45.855675 (+ 4547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:43007: BlockingRecv error: recv got EOF from 127.18.39.67:43007 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:46.243630 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:46.234408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:48357)
0619 06:51:46.234938 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:46.234963 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:46.235077 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:46.235388 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:46.235391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:46.235766 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:46.235928 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.235936 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.236970 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.236974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.237911 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.237922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.238697 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.238705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.238880 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.239684 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:46.239712 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:46.243383 (+ 3671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":294,"threads_started":1}
W20260619 06:51:46.243942 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:46.234481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:48357 (local address 127.18.39.126:38801)
0619 06:51:46.234717 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:46.234722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:46.235047 (+ 325us) server_negotiation.cc:407] Connection header received
0619 06:51:46.235189 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:46.235192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:46.235252 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:46.235339 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:46.236086 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.236826 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.238048 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.238472 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.238570 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.243565 (+ 4995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:48357: BlockingRecv error: recv got EOF from 127.18.39.66:48357 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:46.683710 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:46.674504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:54975)
0619 06:51:46.675033 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:46.675049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:46.675159 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:46.675516 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:46.675521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:46.675933 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:46.676075 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.676084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.677134 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.677138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.678088 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.678100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.678845 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.678850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.678988 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.679648 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:46.679666 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:46.683492 (+ 3826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":293,"threads_started":1}
W20260619 06:51:46.683909 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:46.674525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:54975 (local address 127.18.39.126:38801)
0619 06:51:46.674690 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:46.674695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:46.675125 (+ 430us) server_negotiation.cc:407] Connection header received
0619 06:51:46.675258 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:46.675262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:46.675332 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:46.675418 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:46.676226 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.676984 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.678254 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.678709 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.678815 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.683681 (+ 4866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:54975: BlockingRecv error: recv got EOF from 127.18.39.65:54975 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:46 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:46.865890 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:46.856586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:43223)
0619 06:51:46.857158 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:46.857174 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:46.857289 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:46.857728 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:46.857732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:46.858150 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:46.858318 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.858327 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.859469 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.859473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.860409 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:46.860421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.861160 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.861164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:46.861287 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.862029 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:46.862046 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:46.865529 (+ 3483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":317,"threads_started":1}
W20260619 06:51:46.866026 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:46.856669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:43223 (local address 127.18.39.126:38801)
0619 06:51:46.856835 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:46.856839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:46.857308 (+ 469us) server_negotiation.cc:407] Connection header received
0619 06:51:46.857491 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:46.857495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:46.857562 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:46.857654 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:46.858474 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.859305 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.860562 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:46.860993 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:46.861092 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:46.865767 (+ 4675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:43223: BlockingRecv error: recv got EOF from 127.18.39.67:43223 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:47.254237 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:47.244743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:57445)
0619 06:51:47.245285 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:47.245301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:47.245452 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:47.245823 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:47.245826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:47.246343 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:47.246498 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.246506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.247770 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.247778 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.248672 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.248683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.249511 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.249517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.249700 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.250358 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:47.250375 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:47.253993 (+ 3618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":296,"threads_started":1}
W20260619 06:51:47.254420 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:47.244918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:57445 (local address 127.18.39.126:38801)
0619 06:51:47.245134 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:47.245139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:47.245437 (+ 298us) server_negotiation.cc:407] Connection header received
0619 06:51:47.245558 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:47.245563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:47.245674 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:47.245794 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:47.246660 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.247628 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.248818 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.249281 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.249499 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.254179 (+ 4680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:57445: BlockingRecv error: recv got EOF from 127.18.39.66:57445 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.65@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:47.694206 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:47.685018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.65:45935)
0619 06:51:47.685609 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:47.685631 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:47.685792 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:47.686153 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:47.686157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:47.686555 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:47.686816 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.686829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.687952 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.687958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.688935 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.688946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.689665 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.689671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.689850 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.690441 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:47.690458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:47.693832 (+ 3374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":310,"threads_started":1}
W20260619 06:51:47.694263 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:47.685101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:45935 (local address 127.18.39.126:38801)
0619 06:51:47.685401 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:47.685441 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:47.685727 (+ 286us) server_negotiation.cc:407] Connection header received
0619 06:51:47.685930 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:47.685936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:47.686013 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:47.686119 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:47.686981 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.687791 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.689074 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.689526 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.689654 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.694031 (+ 4377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.65:45935: BlockingRecv error: recv got EOF from 127.18.39.65:45935 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:47 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.67@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:47.876062 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:47.867090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.67:46297)
0619 06:51:47.867643 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:47.867658 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:47.867804 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:47.868233 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:47.868240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:47.868760 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:47.868910 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.868921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.870016 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.870019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.870904 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:47.870916 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.871652 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.871656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:47.871773 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.872339 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:47.872399 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:47.875825 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":315,"threads_started":1}
W20260619 06:51:47.876242 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:47.867142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:46297 (local address 127.18.39.126:38801)
0619 06:51:47.867421 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:47.867430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:47.867734 (+ 304us) server_negotiation.cc:407] Connection header received
0619 06:51:47.867949 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:47.867960 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:47.868052 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:47.868149 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:47.869040 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.869874 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.871029 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:47.871530 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:47.871621 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:47.876006 (+ 4385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.67:46297: BlockingRecv error: recv got EOF from 127.18.39.67:46297 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
Jun 19 06:51:48 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:51:48 dist-test-slave-1h5s krb5kdc[26841](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.39.66@KRBTEST.COM for oryx/127.18.39.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:51:48.264559 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:48.255536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:38801 (local address 127.18.39.66:51303)
0619 06:51:48.255996 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:48.256013 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:48.256130 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:48.256434 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:48.256437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:48.256784 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:48.256975 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:48.256983 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.258119 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.258127 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:48.259386 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:48.259399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.260129 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.260135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:48.260311 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:48.260886 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:48.260902 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:48.264170 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.39.126:38801: Server oryx/127.18.39.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":231,"threads_started":1}
W20260619 06:51:48.264592 27579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:48.255718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:51303 (local address 127.18.39.126:38801)
0619 06:51:48.255942 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:48.255949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:48.256094 (+ 145us) server_negotiation.cc:407] Connection header received
0619 06:51:48.256228 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:48.256231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:48.256307 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:48.256424 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:48.257095 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.257937 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.259518 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.259984 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.260118 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:48.264374 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.39.66:51303: BlockingRecv error: recv got EOF from 127.18.39.66:51303 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 06:51:48.596668 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27430
I20260619 06:51:48.610041 27892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:48.597919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.18.39.65:46517)
0619 06:51:48.598546 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:48.598554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:48.598581 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:51:48.598644 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:48.598650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:48.598878 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:48.599056 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:48.600540 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.601731 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.603009 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.603728 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.603902 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:48.606573 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:48.606582 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:48.606597 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:48.606650 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:48.609487 (+ 2837us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:51:48.609808 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35518: Request ticket server kudu/127.18.39.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":330,"threads_started":1}
W20260619 06:51:48.610162 27892 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35518: Request ticket server kudu/127.18.39.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:51:48.610319 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27430: Not authorized: Client connection negotiation failed: client connection to 127.18.39.65:46517: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:51:48.610615 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27430
I20260619 06:51:48.627650 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27581
I20260619 06:51:48.641225 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:48.628850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.18.39.66:39443)
0619 06:51:48.629619 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:48.629631 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:48.629664 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:51:48.629743 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:48.629755 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:48.630089 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:48.630313 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:48.631719 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.632846 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.634355 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.634968 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.635079 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:48.637952 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:48.637962 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:48.637978 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:48.638023 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:48.640688 (+ 2665us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:51:48.640984 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33448: Request ticket server kudu/127.18.39.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":355,"threads_started":1}
W20260619 06:51:48.641495 27893 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33448: Request ticket server kudu/127.18.39.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:51:48.641577 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27581: Not authorized: Client connection negotiation failed: client connection to 127.18.39.66:39443: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:51:48.641803 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27581
I20260619 06:51:48.659636 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27714
I20260619 06:51:48.673561 27894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:48.660771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.18.39.67:38227)
0619 06:51:48.661426 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:48.661437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:48.661471 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:51:48.661551 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:48.661560 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:48.661907 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:48.662118 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:48.663587 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.665136 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.666365 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:48.666979 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:48.667089 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:48.669544 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:48.669554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:48.669569 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:48.669613 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:48.672750 (+ 3137us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:51:48.673116 (+ 366us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33598: Request ticket server kudu/127.18.39.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":504,"threads_started":1}
W20260619 06:51:48.673589 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27714: Not authorized: Client connection negotiation failed: client connection to 127.18.39.67:38227: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.39.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:51:48.673650 27894 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33598: Request ticket server kudu/127.18.39.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:51:48.673796 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27714
I20260619 06:51:48.690694 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27361
I20260619 06:51:48.692042 27423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:48.806840 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27361
2026-06-19T06:51:48Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21536 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T06:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:48Z Disabled control of system clock
I20260619 06:51:48.884389 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46083
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 06:51:49.216341 27899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:49.216665 27899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:49.216744 27899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:49.223935 27899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:49.224043 27899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:49.224080 27899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:49.224156 27899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:49.224200 27899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:49.233357 27899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:46083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:49.234578 27899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:49.236317 27899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:49.248191 27907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:49.248293 27905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:49.248265 27904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:49.248875 27899 server_base.cc:1034] running on GCE node
I20260619 06:51:49.249715 27899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:49.251116 27899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:49.252393 27899 hybrid_clock.cc:648] HybridClock initialized: now 1781851909252316 us; error 52 us; skew 500 ppm
I20260619 06:51:49.252767 27899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:49.257899 27899 webserver.cc:460] Webserver started at http://127.18.39.126:40815/ using document root <none> and password file <none>
I20260619 06:51:49.258667 27899 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:49.258805 27899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:49.259160 27899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:49.262475 27899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "a33a06d67dee4a62a7fcf65a0f910a0a"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "495c0c14c996ade81321370e51e6c767"
server_key_iv: "f1d6680aa587ec2c6f0d61daca13639d"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.263294 27899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "a33a06d67dee4a62a7fcf65a0f910a0a"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "495c0c14c996ade81321370e51e6c767"
server_key_iv: "f1d6680aa587ec2c6f0d61daca13639d"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.268519 27899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:51:49.271942 27913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:49.273486 27899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:49.273676 27899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "a33a06d67dee4a62a7fcf65a0f910a0a"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "495c0c14c996ade81321370e51e6c767"
server_key_iv: "f1d6680aa587ec2c6f0d61daca13639d"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.273900 27899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:49.293615 27899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:49.294495 27899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:49.294800 27899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:49.313932 27899 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:46083
I20260619 06:51:49.313954 27964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:46083 every 8 connection(s)
I20260619 06:51:49.315342 27899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:49.317247 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27899
I20260619 06:51:49.317478 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:49.318135 18589 external_mini_cluster.cc:1428] Setting key 6376263ee3bc87c2390b1d247bcced4d
I20260619 06:51:49.319793 27965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:49.334030 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:49.320288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.18.39.126:46083)
0619 06:51:49.321422 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:49.321446 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:49.321500 (+ 54us) server_negotiation.cc:407] Connection header received
0619 06:51:49.323281 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:49.323310 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:49.323983 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:49.324477 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:49.325050 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.327148 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.328515 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.329565 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.329744 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:49.331071 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:49.331082 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:49.331103 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:49.331870 (+ 767us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:49.331989 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:49.332022 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:49.332133 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:49.332383 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 06:51:49.332751 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":345,"threads_started":1}
I20260619 06:51:49.334892 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a: Bootstrap starting.
I20260619 06:51:49.338261 27965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:49.339300 27965 log.cc:824] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:49.342497 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a: No bootstrap required, opened a new log
I20260619 06:51:49.350193 27965 raft_consensus.cc:348] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } }
I20260619 06:51:49.350585 27965 raft_consensus.cc:374] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:49.350672 27965 raft_consensus.cc:729] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33a06d67dee4a62a7fcf65a0f910a0a, State: Initialized, Role: FOLLOWER
I20260619 06:51:49.351192 27965 consensus_queue.cc:260] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } }
I20260619 06:51:49.351406 27965 raft_consensus.cc:388] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:49.351497 27965 raft_consensus.cc:482] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:49.351629 27965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:49.353972 27965 raft_consensus.cc:504] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } }
I20260619 06:51:49.354487 27965 leader_election.cc:302] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a33a06d67dee4a62a7fcf65a0f910a0a; no voters:
I20260619 06:51:49.355078 27965 leader_election.cc:288] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:49.355259 27970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:49.356388 27970 raft_consensus.cc:686] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [term 1 LEADER]: Becoming Leader. State: Replica: a33a06d67dee4a62a7fcf65a0f910a0a, State: Running, Role: LEADER
I20260619 06:51:49.357101 27970 consensus_queue.cc:237] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } }
I20260619 06:51:49.357489 27965 sys_catalog.cc:564] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:49.359944 27972 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a33a06d67dee4a62a7fcf65a0f910a0a. Latest consensus state: current_term: 1 leader_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } } }
I20260619 06:51:49.360342 27972 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:49.360855 27971 sys_catalog.cc:455] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a33a06d67dee4a62a7fcf65a0f910a0a" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46083 } } }
I20260619 06:51:49.361279 27971 sys_catalog.cc:458] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:49.364727 27978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:49.370674 27978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:49.379814 27978 catalog_manager.cc:1347] Generated new cluster ID: 9bac2330e47b4a048839a18ddfa20105
I20260619 06:51:49.379951 27978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:49.403260 27978 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:49.404711 27978 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:49.411167 27978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a33a06d67dee4a62a7fcf65a0f910a0a: Generated new TSK 0
I20260619 06:51:49.412091 27978 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:51:49.423012 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--builtin_ntp_servers=127.18.39.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 06:51:49.761070 27989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:49.761438 27989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:49.761596 27989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:49.769310 27989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:49.769510 27989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:49.769681 27989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:49.779359 27989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:49.780687 27989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:49.782604 27989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:49.796092 27997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:49.796133 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:49.796175 27995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:49.797024 27989 server_base.cc:1034] running on GCE node
I20260619 06:51:49.797777 27989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:49.798908 27989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:49.800158 27989 hybrid_clock.cc:648] HybridClock initialized: now 1781851909800099 us; error 71 us; skew 500 ppm
I20260619 06:51:49.800534 27989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:49.802927 27989 webserver.cc:460] Webserver started at http://127.18.39.65:33417/ using document root <none> and password file <none>
I20260619 06:51:49.803676 27989 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:49.803813 27989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:49.804141 27989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:49.807224 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "bed93f1f87944a8ba77cc9fd7162c724"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "1ce72bfad1b0491226d87a35fba5f3e1"
server_key_iv: "62585efb3c354fee0f76e19f1cde9f5a"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.808172 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "bed93f1f87944a8ba77cc9fd7162c724"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "1ce72bfad1b0491226d87a35fba5f3e1"
server_key_iv: "62585efb3c354fee0f76e19f1cde9f5a"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.813694 27989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 06:51:49.817293 28003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:49.818806 27989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:49.819026 27989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "bed93f1f87944a8ba77cc9fd7162c724"
format_stamp: "Formatted at 2026-06-19 06:51:49 on dist-test-slave-1h5s"
server_key: "1ce72bfad1b0491226d87a35fba5f3e1"
server_key_iv: "62585efb3c354fee0f76e19f1cde9f5a"
server_key_version: "encryptionkey@0"
I20260619 06:51:49.819270 27989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:49.834841 27989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:49.835778 27989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:49.836090 27989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:49.837052 27989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:49.838642 27989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:49.838764 27989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:49.838901 27989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:49.838989 27989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:49.872602 27989 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:41733
I20260619 06:51:49.872684 28115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:41733 every 8 connection(s)
I20260619 06:51:49.874191 27989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:49.879026 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 27989
I20260619 06:51:49.879230 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:49.879866 18589 external_mini_cluster.cc:1428] Setting key 36cd01d0fb9a63380cf2501fd18fd9cb
I20260619 06:51:49.882628 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--builtin_ntp_servers=127.18.39.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 06:51:49.887171 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:49.876805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46185 (local address 127.18.39.126:46083)
0619 06:51:49.877419 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:49.877430 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:49.879247 (+ 1817us) server_negotiation.cc:407] Connection header received
0619 06:51:49.880361 (+ 1114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:49.880366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:49.880451 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:49.880578 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:49.881921 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.882626 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.884237 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.884648 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.884784 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:49.886308 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:49.886313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:49.886317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:49.886337 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:49.886378 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:49.886390 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:49.886486 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:49.886722 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 06:51:49.886835 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":363,"threads_started":1}
I20260619 06:51:49.888244 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:49.877275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46083 (local address 127.18.39.65:46185)
0619 06:51:49.878230 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:49.878265 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:49.880095 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:49.880845 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:49.880856 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:49.880985 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:49.881761 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:49.881779 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.882801 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.882805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:49.884082 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:49.884094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:49.884877 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:49.884886 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:49.885135 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:49.885965 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:49.885967 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:49.886011 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:49.886095 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:49.886484 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:49.886496 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:49.886507 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:51:49.886700 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 06:51:49.887148 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":282,"threads_started":1}
I20260619 06:51:49.889954 28116 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46083
I20260619 06:51:49.890277 28116 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:49.891218 28116 heartbeater.cc:511] Master 127.18.39.126:46083 requested a full tablet report, sending...
I20260619 06:51:49.893517 27930 ts_manager.cc:194] Registered new tserver with Master: bed93f1f87944a8ba77cc9fd7162c724 (127.18.39.65:41733)
I20260619 06:51:49.895064 27930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.65:46185
W20260619 06:51:50.214604 28120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:50.214967 28120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:50.215130 28120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:50.222751 28120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:50.222889 28120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:50.223039 28120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:50.232512 28120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:50.233829 28120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:50.235536 28120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:50.248586 28126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:50.248627 28125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:50.248677 28128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:50.249313 28120 server_base.cc:1034] running on GCE node
I20260619 06:51:50.249962 28120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:50.250942 28120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:50.252168 28120 hybrid_clock.cc:648] HybridClock initialized: now 1781851910252118 us; error 47 us; skew 500 ppm
I20260619 06:51:50.252497 28120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:50.254770 28120 webserver.cc:460] Webserver started at http://127.18.39.66:43271/ using document root <none> and password file <none>
I20260619 06:51:50.255510 28120 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:50.255638 28120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:50.255959 28120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:50.258931 28120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "40b95567ebc04e359445e64dc6615778"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "0786a8d8bf75813dde7ac3580f1bdb41"
server_key_iv: "2ae712e3ce13301e894716e102135df3"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.259728 28120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "40b95567ebc04e359445e64dc6615778"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "0786a8d8bf75813dde7ac3580f1bdb41"
server_key_iv: "2ae712e3ce13301e894716e102135df3"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.264825 28120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:51:50.268354 28134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.269794 28120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 06:51:50.269968 28120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "40b95567ebc04e359445e64dc6615778"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "0786a8d8bf75813dde7ac3580f1bdb41"
server_key_iv: "2ae712e3ce13301e894716e102135df3"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.270193 28120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:50.295852 28120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:50.296757 28120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:50.297084 28120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:50.298107 28120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:50.299685 28120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:50.299806 28120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.299922 28120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:50.300006 28120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.338573 28120 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:38585
I20260619 06:51:50.338661 28246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:38585 every 8 connection(s)
I20260619 06:51:50.339862 28120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:50.347218 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28120
I20260619 06:51:50.347383 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:50.347824 18589 external_mini_cluster.cc:1428] Setting key 2dac82f2955fab17f450e9722531f16b
I20260619 06:51:50.350804 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--builtin_ntp_servers=127.18.39.84:44407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 06:51:50.353559 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.342450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:34141 (local address 127.18.39.126:46083)
0619 06:51:50.342622 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:50.342627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:50.344007 (+ 1380us) server_negotiation.cc:407] Connection header received
0619 06:51:50.345827 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:50.345832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:50.345927 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:50.346046 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:50.347277 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.348238 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.350136 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.350833 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.350996 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.352419 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:50.352429 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:50.352438 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:50.352466 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:50.352512 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:50.352532 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:50.352633 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:50.353004 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 06:51:50.353128 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:51:50.355783 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.342979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46083 (local address 127.18.39.66:34141)
0619 06:51:50.343856 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:50.343891 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:50.345561 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:50.346270 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:50.346279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:50.346372 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:50.347076 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:50.347099 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.348452 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.348457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:50.349982 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:50.349994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.351091 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.351097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:50.351338 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.352070 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:50.352073 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:50.352113 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:50.352196 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:50.352701 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:50.352713 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:50.352723 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:51:50.353773 (+ 1050us) client_negotiation.cc:240] Negotiation successful
0619 06:51:50.354246 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":264,"threads_started":1}
I20260619 06:51:50.357681 28247 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46083
I20260619 06:51:50.358093 28247 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:50.359162 28247 heartbeater.cc:511] Master 127.18.39.126:46083 requested a full tablet report, sending...
I20260619 06:51:50.361032 27930 ts_manager.cc:194] Registered new tserver with Master: 40b95567ebc04e359445e64dc6615778 (127.18.39.66:38585)
I20260619 06:51:50.361963 27930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.66:34141
W20260619 06:51:50.688023 28250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:50.688309 28250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:50.688395 28250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:50.695624 28250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:50.695729 28250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:50.695840 28250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:50.705618 28250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46083
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:50.706869 28250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:50.708665 28250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:50.722200 28255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:50.722234 28258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:50.722316 28256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:50.723107 28250 server_base.cc:1034] running on GCE node
I20260619 06:51:50.723743 28250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:50.724772 28250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:50.725972 28250 hybrid_clock.cc:648] HybridClock initialized: now 1781851910725910 us; error 52 us; skew 500 ppm
I20260619 06:51:50.726315 28250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:50.728574 28250 webserver.cc:460] Webserver started at http://127.18.39.67:36531/ using document root <none> and password file <none>
I20260619 06:51:50.729274 28250 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:50.729442 28250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:50.729797 28250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:50.732724 28250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "11d177323cf74c27a18b44bb89a6d579"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "3cb61e6af741ab0f43e9e6fcaac2f703"
server_key_iv: "c2417166460e69f19d1f0b040778806e"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.733613 28250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "11d177323cf74c27a18b44bb89a6d579"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "3cb61e6af741ab0f43e9e6fcaac2f703"
server_key_iv: "c2417166460e69f19d1f0b040778806e"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.738777 28250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:51:50.742267 28264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.743647 28250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:51:50.743836 28250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "11d177323cf74c27a18b44bb89a6d579"
format_stamp: "Formatted at 2026-06-19 06:51:50 on dist-test-slave-1h5s"
server_key: "3cb61e6af741ab0f43e9e6fcaac2f703"
server_key_iv: "c2417166460e69f19d1f0b040778806e"
server_key_version: "encryptionkey@0"
I20260619 06:51:50.744064 28250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:50.780661 28250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:50.781617 28250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:50.781886 28250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:50.782826 28250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:50.784327 28250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:50.784411 28250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.784487 28250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:50.784541 28250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:50.818274 28250 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44513
I20260619 06:51:50.818334 28376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44513 every 8 connection(s)
I20260619 06:51:50.819651 28250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:50.819968 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28250
I20260619 06:51:50.820122 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:50.820794 18589 external_mini_cluster.cc:1428] Setting key 169c3440dd6b812569c3ccd680e8dd29
I20260619 06:51:50.832949 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.822849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:36881 (local address 127.18.39.126:46083)
0619 06:51:50.823083 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:50.823093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:50.824866 (+ 1773us) server_negotiation.cc:407] Connection header received
0619 06:51:50.826616 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:50.826620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:50.826684 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:50.826766 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:50.827814 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.828561 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.830259 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.830657 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.830759 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.832249 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:50.832254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:50.832257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:50.832277 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:50.832306 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:50.832317 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:50.832388 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:50.832611 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 06:51:50.832691 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 06:51:50.834128 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.823608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46083 (local address 127.18.39.67:36881)
0619 06:51:50.824729 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:50.824765 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:50.826370 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:50.826960 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:50.826968 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:50.827057 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:50.827665 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:50.827682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.828688 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.828691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:50.830107 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:50.830124 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.830887 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.830892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:50.831118 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.831924 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:50.831927 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:50.831976 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:50.832060 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:50.832419 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:50.832426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:50.832432 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:51:50.832593 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 06:51:50.832902 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":353,"threads_started":1}
I20260619 06:51:50.835582 28377 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46083
I20260619 06:51:50.835942 28377 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:50.836980 28377 heartbeater.cc:511] Master 127.18.39.126:46083 requested a full tablet report, sending...
I20260619 06:51:50.838645 27930 ts_manager.cc:194] Registered new tserver with Master: 11d177323cf74c27a18b44bb89a6d579 (127.18.39.67:44513)
I20260619 06:51:50.839532 27930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.67:36881
I20260619 06:51:50.849924 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 06:51:50.856304 28118 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:50.854604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.18.39.126:46083)
0619 06:51:50.854813 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:50.854822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:50.855194 (+ 372us) server_negotiation.cc:407] Connection header received
0619 06:51:50.855394 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:50.855399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:50.855495 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:50.855601 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:50.856039 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57352: BlockingRecv error: recv got EOF from 127.0.0.1:57352 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:51:50.856904 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27989
I20260619 06:51:50.864902 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.857996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.18.39.65:41733)
0619 06:51:50.858605 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:50.858620 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:50.858647 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:51:50.858724 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:50.858734 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:50.858953 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:50.859083 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:50.859485 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.860465 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.861725 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.862276 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.862373 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.863402 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:50.863411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:50.863424 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:50.864019 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:50.864067 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:50.864102 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:50.864196 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:50.864461 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 06:51:50.864605 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":312,"threads_started":1}
I20260619 06:51:50.866230 28111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:50.946311 28116 heartbeater.cc:503] Master 127.18.39.126:46083 was elected leader, sending a full tablet report...
I20260619 06:51:50.949520 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27989
I20260619 06:51:50.965848 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28120
I20260619 06:51:50.973968 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:50.966763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.18.39.66:38585)
0619 06:51:50.967299 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:50.967307 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:50.967334 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:51:50.967392 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:50.967398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:50.967621 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:50.967793 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:50.968200 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.969205 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.970485 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:50.971064 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:50.971207 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:50.972330 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:50.972338 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:50.972352 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:50.972973 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:50.973020 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:50.973038 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:50.973164 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:50.973485 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 06:51:50.973620 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":269,"threads_started":1}
I20260619 06:51:50.975338 28242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:51.055904 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28120
I20260619 06:51:51.071954 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28250
I20260619 06:51:51.080096 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:51.073025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.18.39.67:44513)
0619 06:51:51.073767 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:51.073774 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:51.073805 (+ 31us) server_negotiation.cc:407] Connection header received
0619 06:51:51.073863 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:51.073869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:51.074096 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:51.074229 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:51.074595 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:51.075634 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:51.076852 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:51.077495 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:51.077590 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:51.078671 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:51.078679 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:51.078692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:51.079289 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:51.079347 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:51.079367 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:51.079459 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:51.079586 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 06:51:51.079731 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":452,"threads_started":1}
I20260619 06:51:51.081385 28372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:51.165551 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28250
I20260619 06:51:51.181762 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 27899
I20260619 06:51:51.183048 27960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:51.285894 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 27899
2026-06-19T06:51:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2473 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T06:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:51Z Disabled control of system clock
I20260619 06:51:51.359227 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41749
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:41749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 06:51:51.690174 28396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:51.690470 28396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:51.690554 28396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:51.697768 28396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:51.697886 28396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:51.697922 28396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:51.697985 28396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:51.698028 28396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:51.707191 28396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:41749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:51.708442 28396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:51.710171 28396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:51.722275 28401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:51.722193 28404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:51.722153 28402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:51.723193 28396 server_base.cc:1034] running on GCE node
I20260619 06:51:51.724087 28396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:51.725667 28396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:51.726902 28396 hybrid_clock.cc:648] HybridClock initialized: now 1781851911726844 us; error 68 us; skew 500 ppm
I20260619 06:51:51.727267 28396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:51.729691 28396 webserver.cc:460] Webserver started at http://127.18.39.126:44375/ using document root <none> and password file <none>
I20260619 06:51:51.730360 28396 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:51.730458 28396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:51.730734 28396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:51.733712 28396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "dec3843c4394409192bd8d77679160b5"
format_stamp: "Formatted at 2026-06-19 06:51:51 on dist-test-slave-1h5s"
server_key: "fea4a87be55e0fdc0bf7d5eb000d1341"
server_key_iv: "6b5def8055206d88f3824f2b7d6a38d8"
server_key_version: "encryptionkey@0"
I20260619 06:51:51.734457 28396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "dec3843c4394409192bd8d77679160b5"
format_stamp: "Formatted at 2026-06-19 06:51:51 on dist-test-slave-1h5s"
server_key: "fea4a87be55e0fdc0bf7d5eb000d1341"
server_key_iv: "6b5def8055206d88f3824f2b7d6a38d8"
server_key_version: "encryptionkey@0"
I20260619 06:51:51.739626 28396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:51.743187 28410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:51.744659 28396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:51:51.744848 28396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "dec3843c4394409192bd8d77679160b5"
format_stamp: "Formatted at 2026-06-19 06:51:51 on dist-test-slave-1h5s"
server_key: "fea4a87be55e0fdc0bf7d5eb000d1341"
server_key_iv: "6b5def8055206d88f3824f2b7d6a38d8"
server_key_version: "encryptionkey@0"
I20260619 06:51:51.745055 28396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:51.751005 28396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:51.751860 28396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:51.752174 28396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:51.770593 28396 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:41749
I20260619 06:51:51.770632 28461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:41749 every 8 connection(s)
I20260619 06:51:51.772039 28396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:51.775875 28462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:51.781872 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28396
I20260619 06:51:51.782099 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:51.782721 18589 external_mini_cluster.cc:1428] Setting key d48e8251cf7425f621ddffc12a27396b
I20260619 06:51:51.787431 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5: Bootstrap starting.
I20260619 06:51:51.790916 28462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:51.791404 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:51.784866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44628 (local address 127.18.39.126:41749)
0619 06:51:51.785923 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:51.785943 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:51.785983 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:51:51.787782 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:51.787818 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:51.788245 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:51.788622 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:51.789730 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:51.789737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:51.789749 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:51.789763 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:51.789821 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:51.789836 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:51.790067 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:51.790201 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 06:51:51.790438 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":375,"threads_started":1}
I20260619 06:51:51.792273 28462 log.cc:824] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:51.795328 28462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5: No bootstrap required, opened a new log
I20260619 06:51:51.802536 28462 raft_consensus.cc:348] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } }
I20260619 06:51:51.802862 28462 raft_consensus.cc:374] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:51.802932 28462 raft_consensus.cc:729] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dec3843c4394409192bd8d77679160b5, State: Initialized, Role: FOLLOWER
I20260619 06:51:51.803524 28462 consensus_queue.cc:260] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } }
I20260619 06:51:51.803718 28462 raft_consensus.cc:388] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:51.803812 28462 raft_consensus.cc:482] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:51.803967 28462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:51.806521 28462 raft_consensus.cc:504] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } }
I20260619 06:51:51.807057 28462 leader_election.cc:302] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dec3843c4394409192bd8d77679160b5; no voters:
I20260619 06:51:51.807647 28462 leader_election.cc:288] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:51.807798 28467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:51.808544 28467 raft_consensus.cc:686] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [term 1 LEADER]: Becoming Leader. State: Replica: dec3843c4394409192bd8d77679160b5, State: Running, Role: LEADER
I20260619 06:51:51.809073 28467 consensus_queue.cc:237] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } }
I20260619 06:51:51.809734 28462 sys_catalog.cc:564] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:51.811883 28468 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dec3843c4394409192bd8d77679160b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } } }
I20260619 06:51:51.812250 28468 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:51.812450 28469 sys_catalog.cc:455] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dec3843c4394409192bd8d77679160b5. Latest consensus state: current_term: 1 leader_uuid: "dec3843c4394409192bd8d77679160b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dec3843c4394409192bd8d77679160b5" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41749 } } }
I20260619 06:51:51.812834 28469 sys_catalog.cc:458] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5 [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:51.813900 28473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:51.819370 28473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:51.828665 28473 catalog_manager.cc:1347] Generated new cluster ID: 71988e9438264a86841dd9352d49c401
I20260619 06:51:51.828799 28473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:51.867383 28473 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:51.867842 28473 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:51.877764 28473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dec3843c4394409192bd8d77679160b5: Generated new TSK 0
I20260619 06:51:51.878631 28473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:51:51.891268 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--builtin_ntp_servers=127.18.39.84:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 06:51:52.243273 28486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:52.243577 28486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:52.243744 28486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:52.251483 28486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:52.251607 28486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:52.251791 28486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:52.261582 28486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:52.263006 28486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:52.264838 28486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:52.278215 28491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:52.278231 28494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:52.278286 28492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:52.279171 28486 server_base.cc:1034] running on GCE node
I20260619 06:51:52.279819 28486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:52.280871 28486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:52.282073 28486 hybrid_clock.cc:648] HybridClock initialized: now 1781851912282028 us; error 31 us; skew 500 ppm
I20260619 06:51:52.282455 28486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:52.284824 28486 webserver.cc:460] Webserver started at http://127.18.39.65:43847/ using document root <none> and password file <none>
I20260619 06:51:52.285621 28486 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:52.285735 28486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:52.286072 28486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:52.288987 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "88cce33536d147e7b98bd0346cc6830a"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "13e96ad030bb108866462597851ecbcf"
server_key_iv: "cc9518241fe661d925b8e05b493f66b9"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.289911 28486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "88cce33536d147e7b98bd0346cc6830a"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "13e96ad030bb108866462597851ecbcf"
server_key_iv: "cc9518241fe661d925b8e05b493f66b9"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.295338 28486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:52.298976 28500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:52.300457 28486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:51:52.300663 28486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "88cce33536d147e7b98bd0346cc6830a"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "13e96ad030bb108866462597851ecbcf"
server_key_iv: "cc9518241fe661d925b8e05b493f66b9"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.300910 28486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:52.306851 28486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:52.307718 28486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:52.308002 28486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:52.308941 28486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:52.310534 28486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:52.310662 28486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:52.310791 28486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:52.310947 28486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 06:51:52.344498 28486 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:39681
I20260619 06:51:52.344547 28612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:39681 every 8 connection(s)
I20260619 06:51:52.346158 28486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:52.347520 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28486
I20260619 06:51:52.347723 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:52.348505 18589 external_mini_cluster.cc:1428] Setting key 39c340fa1a913aa24c6c0fbdaf34e1e5
I20260619 06:51:52.351518 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--builtin_ntp_servers=127.18.39.84:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 06:51:52.361568 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:52.349314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:55507 (local address 127.18.39.126:41749)
0619 06:51:52.349808 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:52.349813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:52.351406 (+ 1593us) server_negotiation.cc:407] Connection header received
0619 06:51:52.353765 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:52.353773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:52.353861 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:52.353982 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:52.360632 (+ 6650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:52.360645 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:52.360649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:52.360670 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:52.360715 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:52.360727 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:52.360816 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:52.361166 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 06:51:52.361239 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":264,"threads_started":1}
I20260619 06:51:52.362959 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:52.350069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41749 (local address 127.18.39.65:55507)
0619 06:51:52.351213 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:52.351264 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:52.353411 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:52.354384 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:52.354399 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:52.354547 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:52.360026 (+ 5479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:52.360030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:52.360121 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:52.360319 (+ 198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:52.360884 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:52.360893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:52.360942 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 06:51:52.361262 (+ 320us) client_negotiation.cc:240] Negotiation successful
0619 06:51:52.361617 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":366,"threads_started":1}
I20260619 06:51:52.366546 28613 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41749
I20260619 06:51:52.366947 28613 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:52.367656 28613 heartbeater.cc:511] Master 127.18.39.126:41749 requested a full tablet report, sending...
I20260619 06:51:52.370519 28427 ts_manager.cc:194] Registered new tserver with Master: 88cce33536d147e7b98bd0346cc6830a (127.18.39.65:39681)
W20260619 06:51:52.688723 28617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:52.689028 28617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:52.689121 28617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:52.696410 28617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:52.696528 28617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:52.696642 28617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:52.705804 28617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:52.707072 28617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:52.708842 28617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:52.722329 28623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:52.722470 28622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:52.722498 28625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:52.723630 28617 server_base.cc:1034] running on GCE node
I20260619 06:51:52.724282 28617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:52.725272 28617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:52.726545 28617 hybrid_clock.cc:648] HybridClock initialized: now 1781851912726497 us; error 40 us; skew 500 ppm
I20260619 06:51:52.726881 28617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:52.729120 28617 webserver.cc:460] Webserver started at http://127.18.39.66:46357/ using document root <none> and password file <none>
I20260619 06:51:52.729903 28617 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:52.730034 28617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:52.730355 28617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:52.733568 28617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "0fcddacd4d974593a177812f9de7bcc7"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "76a034ed6cc58853b6f5c93e03b1e206"
server_key_iv: "06026e5fa74feb8f92ad658129e05458"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.734521 28617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "0fcddacd4d974593a177812f9de7bcc7"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "76a034ed6cc58853b6f5c93e03b1e206"
server_key_iv: "06026e5fa74feb8f92ad658129e05458"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.739853 28617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:51:52.743530 28631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:52.744947 28617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:51:52.745138 28617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "0fcddacd4d974593a177812f9de7bcc7"
format_stamp: "Formatted at 2026-06-19 06:51:52 on dist-test-slave-1h5s"
server_key: "76a034ed6cc58853b6f5c93e03b1e206"
server_key_iv: "06026e5fa74feb8f92ad658129e05458"
server_key_version: "encryptionkey@0"
I20260619 06:51:52.745389 28617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:52.751367 28617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:52.752204 28617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:52.752475 28617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:52.753485 28617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:52.755110 28617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:52.755235 28617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:52.755366 28617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:52.755463 28617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:52.789875 28617 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:38537
I20260619 06:51:52.789912 28743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:38537 every 8 connection(s)
I20260619 06:51:52.791219 28617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:52.793548 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28617
I20260619 06:51:52.793740 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:52.794322 18589 external_mini_cluster.cc:1428] Setting key 5c8a1ec746efa2799cdfe314299bc82c
I20260619 06:51:52.797693 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--builtin_ntp_servers=127.18.39.84:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 06:51:52.801122 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:52.793974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:37851 (local address 127.18.39.126:41749)
0619 06:51:52.794194 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:52.794200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:52.795963 (+ 1763us) server_negotiation.cc:407] Connection header received
0619 06:51:52.798604 (+ 2641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:52.798609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:52.798667 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:52.798746 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:52.800398 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:52.800403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:52.800406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:52.800422 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:52.800452 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:52.800463 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:52.800565 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:52.800850 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 06:51:52.800907 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:51:52.802565 28746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:52.794650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41749 (local address 127.18.39.66:37851)
0619 06:51:52.795802 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:52.795847 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:52.798234 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:52.799072 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:52.799089 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:52.799236 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:52.800012 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:52.800017 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:52.800070 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:52.800191 (+ 121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:52.800574 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:52.800587 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:52.800597 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:51:52.800900 (+ 303us) client_negotiation.cc:240] Negotiation successful
0619 06:51:52.801200 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":420,"threads_started":1}
I20260619 06:51:52.804140 28744 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41749
I20260619 06:51:52.804512 28744 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:52.805361 28744 heartbeater.cc:511] Master 127.18.39.126:41749 requested a full tablet report, sending...
I20260619 06:51:52.807220 28427 ts_manager.cc:194] Registered new tserver with Master: 0fcddacd4d974593a177812f9de7bcc7 (127.18.39.66:38537)
W20260619 06:51:53.133590 28747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:53.133905 28747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:53.133996 28747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:53.141726 28747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:53.141840 28747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:53.141952 28747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:53.151437 28747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:53.152782 28747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:53.154588 28747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:53.167654 28753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:53.167704 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:53.167726 28752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:53.168503 28747 server_base.cc:1034] running on GCE node
I20260619 06:51:53.169078 28747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:53.170056 28747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:53.171253 28747 hybrid_clock.cc:648] HybridClock initialized: now 1781851913171215 us; error 38 us; skew 500 ppm
I20260619 06:51:53.171588 28747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:51:53.173945 28747 webserver.cc:460] Webserver started at http://127.18.39.67:35493/ using document root <none> and password file <none>
I20260619 06:51:53.174633 28747 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:53.174718 28747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:53.175078 28747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:53.177934 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "c9bad882fc504152bf9d0820725b5805"
format_stamp: "Formatted at 2026-06-19 06:51:53 on dist-test-slave-1h5s"
server_key: "99233ddc81b9874afcd856d0b780e6e4"
server_key_iv: "45079fc3e97debbd5346e4278a15a911"
server_key_version: "encryptionkey@0"
I20260619 06:51:53.178709 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "c9bad882fc504152bf9d0820725b5805"
format_stamp: "Formatted at 2026-06-19 06:51:53 on dist-test-slave-1h5s"
server_key: "99233ddc81b9874afcd856d0b780e6e4"
server_key_iv: "45079fc3e97debbd5346e4278a15a911"
server_key_version: "encryptionkey@0"
I20260619 06:51:53.183671 28747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:53.187094 28761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:53.188403 28747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:51:53.188582 28747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "c9bad882fc504152bf9d0820725b5805"
format_stamp: "Formatted at 2026-06-19 06:51:53 on dist-test-slave-1h5s"
server_key: "99233ddc81b9874afcd856d0b780e6e4"
server_key_iv: "45079fc3e97debbd5346e4278a15a911"
server_key_version: "encryptionkey@0"
I20260619 06:51:53.188805 28747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:53.194373 28747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:53.195132 28747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:53.195398 28747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:53.196266 28747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:53.197666 28747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:53.197786 28747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:53.197921 28747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:53.198014 28747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:53.230911 28747 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:38873
I20260619 06:51:53.230997 28873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:38873 every 8 connection(s)
I20260619 06:51:53.232213 28747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:53.233253 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28747
I20260619 06:51:53.233503 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:53.234028 18589 external_mini_cluster.cc:1428] Setting key b30917f6ab93ad60d6f27cfa9daaccce
I20260619 06:51:53.240603 28615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:53.235132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:59731 (local address 127.18.39.126:41749)
0619 06:51:53.235289 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:53.235293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:53.236909 (+ 1616us) server_negotiation.cc:407] Connection header received
0619 06:51:53.238646 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:53.238650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:53.238702 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:53.238790 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:53.239963 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:53.239967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:53.239970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:53.239985 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:53.240013 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:53.240024 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:53.240117 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:53.240295 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 06:51:53.240392 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 06:51:53.241506 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:53.235685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41749 (local address 127.18.39.67:59731)
0619 06:51:53.236762 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:53.236805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:53.238394 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:53.238982 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:53.238991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:53.239098 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:53.239695 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:51:53.239697 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:53.239733 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:51:53.239816 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:53.240090 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:53.240095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:53.240100 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:51:53.240308 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 06:51:53.240500 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":364,"threads_started":1}
I20260619 06:51:53.242808 28874 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41749
I20260619 06:51:53.243100 28874 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:53.243633 28874 heartbeater.cc:511] Master 127.18.39.126:41749 requested a full tablet report, sending...
I20260619 06:51:53.244944 28427 ts_manager.cc:194] Registered new tserver with Master: c9bad882fc504152bf9d0820725b5805 (127.18.39.67:38873)
I20260619 06:51:53.249492 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 06:51:53.255904 28615 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:51:53.254213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.18.39.126:41749)
0619 06:51:53.254453 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:53.254462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:53.254937 (+ 475us) server_negotiation.cc:407] Connection header received
0619 06:51:53.255155 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:53.255159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:53.255215 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:53.255318 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:53.255671 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44636: BlockingRecv error: recv got EOF from 127.0.0.1:44636 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:51:53.256628 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28486
I20260619 06:51:53.260114 28883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:53.257642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.18.39.65:39681)
0619 06:51:53.258176 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:53.258184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:53.258213 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:51:53.258279 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:53.258286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:53.258553 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:53.258689 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:53.259540 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:53.259546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:53.259557 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:53.259569 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:53.259611 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:53.259628 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:53.259730 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:53.259810 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 06:51:53.259899 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":263,"threads_started":1}
I20260619 06:51:53.261408 28608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:53.342406 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28486
I20260619 06:51:53.358670 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28617
I20260619 06:51:53.362308 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:53.359653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.18.39.66:38537)
0619 06:51:53.360145 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:53.360153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:53.360180 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:51:53.360246 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:53.360253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:53.360474 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:53.360603 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:53.361558 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:53.361565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:53.361578 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:53.361592 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:53.361643 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:53.361662 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:53.361769 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:53.361911 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 06:51:53.362023 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":238,"threads_started":1}
I20260619 06:51:53.363481 28739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:53.447737 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28617
I20260619 06:51:53.465119 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28747
I20260619 06:51:53.469251 28887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:53.466233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.18.39.67:38873)
0619 06:51:53.466859 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:53.466866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:53.466895 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:51:53.466956 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:53.466962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:53.467176 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:53.467342 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:53.468518 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:53.468524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:53.468535 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:51:53.468551 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:53.468588 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:51:53.468601 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:53.468783 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:53.468858 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 06:51:53.468956 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":320,"threads_started":1}
I20260619 06:51:53.470613 28869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:53.552914 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28747
I20260619 06:51:53.568163 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28396
I20260619 06:51:53.569482 28457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:53.671733 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28396
2026-06-19T06:51:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2385 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:53 dist-test-slave-1h5s krb5kdc[28892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:53 dist-test-slave-1h5s krb5kdc[28892](info): set up 2 sockets
Jun 19 06:51:53 dist-test-slave-1h5s krb5kdc[28892](info): commencing operation
krb5kdc: starting...
W20260619 06:51:54.782538 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:54 dist-test-slave-1h5s krb5kdc[28892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:51:55.020401 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46547
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:55.355213 28905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:55.355594 28905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:55.355746 28905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:55.363097 28905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:51:55.363260 28905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:55.363372 28905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:55.363483 28905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:51:55.363559 28905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:51:55.373740 28905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:46547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:46547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:55.375157 28905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:55.376494 28905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:55.388618 28910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:55.388635 28911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:55.388682 28913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:55.389721 28905 server_base.cc:1034] running on GCE node
I20260619 06:51:55.390623 28905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:55.392215 28905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:55.393440 28905 hybrid_clock.cc:648] HybridClock initialized: now 1781851915393334 us; error 85 us; skew 500 ppm
I20260619 06:51:55.393790 28905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:55 dist-test-slave-1h5s krb5kdc[28892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851915, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:55.396813 28905 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:51:55.398274 28905 webserver.cc:460] Webserver started at http://127.18.39.126:37309/ using document root <none> and password file <none>
I20260619 06:51:55.398922 28905 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:55.399019 28905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:55.399272 28905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:55.402310 28905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "d934839ec87c4c5bb34a913625c04cfe"
format_stamp: "Formatted at 2026-06-19 06:51:55 on dist-test-slave-1h5s"
server_key: "833ac1cd6ed600f6a23d2976f726caaf"
server_key_iv: "d50d807d1fe00037252a5ce391e8a933"
server_key_version: "encryptionkey@0"
I20260619 06:51:55.403044 28905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "d934839ec87c4c5bb34a913625c04cfe"
format_stamp: "Formatted at 2026-06-19 06:51:55 on dist-test-slave-1h5s"
server_key: "833ac1cd6ed600f6a23d2976f726caaf"
server_key_iv: "d50d807d1fe00037252a5ce391e8a933"
server_key_version: "encryptionkey@0"
I20260619 06:51:55.408082 28905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:51:55.411509 28920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:55.412956 28905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:51:55.413092 28905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "d934839ec87c4c5bb34a913625c04cfe"
format_stamp: "Formatted at 2026-06-19 06:51:55 on dist-test-slave-1h5s"
server_key: "833ac1cd6ed600f6a23d2976f726caaf"
server_key_iv: "d50d807d1fe00037252a5ce391e8a933"
server_key_version: "encryptionkey@0"
I20260619 06:51:55.413254 28905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:55.429291 28905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:55.433910 28905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:55.434166 28905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:55.454371 28905 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:46547
I20260619 06:51:55.454409 28971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:46547 every 8 connection(s)
I20260619 06:51:55.455826 28905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:51:55.459774 28972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:55.463451 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28905
I20260619 06:51:55.463701 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:51:55.464349 18589 external_mini_cluster.cc:1428] Setting key a910ebe744fc2adc8817035cdd0ce085
I20260619 06:51:55.470980 28972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: Bootstrap starting.
I20260619 06:51:55.474783 28972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:55.475984 28972 log.cc:824] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:55.479560 28972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: No bootstrap required, opened a new log
Jun 19 06:51:55 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:55.487905 28972 raft_consensus.cc:348] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } }
I20260619 06:51:55.488404 28972 raft_consensus.cc:374] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:55.488534 28972 raft_consensus.cc:729] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d934839ec87c4c5bb34a913625c04cfe, State: Initialized, Role: FOLLOWER
I20260619 06:51:55.489269 28972 consensus_queue.cc:260] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } }
I20260619 06:51:55.489537 28975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:55.466395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.18.39.126:46547)
0619 06:51:55.467200 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:55.467212 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:55.467253 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:51:55.469495 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:55.469523 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:55.469932 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:55.470280 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:55.472195 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:55.473609 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:55.475485 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:55.476240 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:55.476391 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:55.481135 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:55.481146 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:55.481164 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:55.481204 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:55.484801 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:55.485679 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:55.485687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:55.485694 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:55.485869 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:55.486276 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:55.486285 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:55.486291 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:55.486899 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:55.487240 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:55.487678 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 06:51:55.488142 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":380,"threads_started":1}
I20260619 06:51:55.489588 28972 raft_consensus.cc:388] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:51:55.489722 28972 raft_consensus.cc:482] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:51:55.489905 28972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:55.492290 28972 raft_consensus.cc:504] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } }
I20260619 06:51:55.492846 28972 leader_election.cc:302] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d934839ec87c4c5bb34a913625c04cfe; no voters:
I20260619 06:51:55.493466 28972 leader_election.cc:288] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:51:55.493641 28977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:55.494333 28977 raft_consensus.cc:686] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [term 1 LEADER]: Becoming Leader. State: Replica: d934839ec87c4c5bb34a913625c04cfe, State: Running, Role: LEADER
I20260619 06:51:55.494880 28977 consensus_queue.cc:237] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } }
I20260619 06:51:55.495635 28972 sys_catalog.cc:564] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:51:55.497823 28978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d934839ec87c4c5bb34a913625c04cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } } }
I20260619 06:51:55.498127 28978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:55.498874 28979 sys_catalog.cc:455] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader d934839ec87c4c5bb34a913625c04cfe. Latest consensus state: current_term: 1 leader_uuid: "d934839ec87c4c5bb34a913625c04cfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d934839ec87c4c5bb34a913625c04cfe" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 46547 } } }
I20260619 06:51:55.499164 28979 sys_catalog.cc:458] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe [sys.catalog]: This master's current role is: LEADER
I20260619 06:51:55.499985 28983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:51:55.506579 28983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:51:55.515734 28983 catalog_manager.cc:1347] Generated new cluster ID: e11bcc384dda4a89bd403181acea83b4
I20260619 06:51:55.515867 28983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:51:55.540925 28983 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:51:55.542372 28983 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:51:55.558211 28983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: Generated new TSK 0
I20260619 06:51:55.559110 28983 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:51:55.625066 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:55.962455 28998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:55.962806 28998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:55.962978 28998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:55.970484 28998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:55.970643 28998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:55.970824 28998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:51:55.980895 28998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:55.982599 28998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:55.983989 28998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:55.997745 29003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:55.997936 29004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:55.997745 29006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:56.003976 28998 server_base.cc:1034] running on GCE node
I20260619 06:51:56.004690 28998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:56.005841 28998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:56.007081 28998 hybrid_clock.cc:648] HybridClock initialized: now 1781851916007035 us; error 50 us; skew 500 ppm
I20260619 06:51:56.007506 28998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:56 dist-test-slave-1h5s krb5kdc[28892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851916, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:56.010982 28998 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:51:56.012638 28998 webserver.cc:460] Webserver started at http://127.18.39.65:35207/ using document root <none> and password file <none>
I20260619 06:51:56.013459 28998 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:56.013604 28998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:56.013916 28998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:56.016963 28998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "fa2258f13b2f4623a506c3a5d8f70269"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "260104adfd79c5160bdcbb0798cc84ba"
server_key_iv: "ed3a47e580fe651e72143035cfa95b55"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.017774 28998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "fa2258f13b2f4623a506c3a5d8f70269"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "260104adfd79c5160bdcbb0798cc84ba"
server_key_iv: "ed3a47e580fe651e72143035cfa95b55"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.023276 28998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:51:56.026961 29013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:56.028312 28998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:51:56.028527 28998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "fa2258f13b2f4623a506c3a5d8f70269"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "260104adfd79c5160bdcbb0798cc84ba"
server_key_iv: "ed3a47e580fe651e72143035cfa95b55"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.028738 28998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:56.050577 28998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:56.054642 28998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:56.054952 28998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:56.055985 28998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:56.057405 28998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:56.057507 28998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:56.057658 28998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:56.057739 28998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:56.091249 28998 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:35361
I20260619 06:51:56.091305 29125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:35361 every 8 connection(s)
I20260619 06:51:56.092541 28998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:51:56.100885 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 28998
I20260619 06:51:56.101143 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:51:56.101851 18589 external_mini_cluster.cc:1428] Setting key 0c2b2e87d753ef3c21f6912db2e6ae90
Jun 19 06:51:56 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851916, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:56.114715 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:56.095032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:47987 (local address 127.18.39.126:46547)
0619 06:51:56.095478 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:56.095484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:56.096666 (+ 1182us) server_negotiation.cc:407] Connection header received
0619 06:51:56.098524 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:56.098528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:56.098592 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:56.098694 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:56.100221 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.101301 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.103489 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.104097 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.104239 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:56.107766 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:56.107775 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:56.107783 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:56.107829 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:56.111416 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:56.112315 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:56.112319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:56.112321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:56.112446 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:56.112898 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:56.112904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:56.112907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:56.113187 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:56.113375 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:56.114118 (+ 743us) server_negotiation.cc:299] Negotiation successful
0619 06:51:56.114352 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":202,"threads_started":1}
I20260619 06:51:56.115751 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:56.095645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46547 (local address 127.18.39.65:47987)
0619 06:51:56.096506 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:56.096552 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:56.098275 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:56.098940 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:56.098948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:56.099494 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:56.100070 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:56.100086 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.101463 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.101467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:56.103307 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:56.103322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.104319 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.104324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:56.104536 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:56.105259 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:56.105286 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:56.107512 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:56.111604 (+ 4092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:56.111615 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:56.111638 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:56.112113 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:56.112569 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:56.112574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:56.112578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:56.112782 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:56.113322 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:56.113355 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:56.113737 (+ 382us) client_negotiation.cc:769] Sending connection context
0619 06:51:56.114024 (+ 287us) client_negotiation.cc:240] Negotiation successful
0619 06:51:56.114596 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":365,"threads_started":1}
I20260619 06:51:56.117462 29126 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46547
I20260619 06:51:56.117782 29126 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:56.118537 29126 heartbeater.cc:511] Master 127.18.39.126:46547 requested a full tablet report, sending...
I20260619 06:51:56.120831 28937 ts_manager.cc:194] Registered new tserver with Master: fa2258f13b2f4623a506c3a5d8f70269 (127.18.39.65:35361)
I20260619 06:51:56.123159 28937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:47987
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:51:56.153780 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:56.488885 29132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:56.489233 29132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:56.489459 29132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:56.496848 29132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:56.497017 29132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:56.497210 29132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:51:56.507252 29132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:56.508638 29132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:56.509987 29132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:56.523443 29137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:56.523865 29140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:56.523414 29138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:56.524286 29132 server_base.cc:1034] running on GCE node
I20260619 06:51:56.524981 29132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:56.526090 29132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:56.527324 29132 hybrid_clock.cc:648] HybridClock initialized: now 1781851916527259 us; error 62 us; skew 500 ppm
I20260619 06:51:56.527659 29132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:56 dist-test-slave-1h5s krb5kdc[28892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851916, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:56.530913 29132 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:51:56.532408 29132 webserver.cc:460] Webserver started at http://127.18.39.66:39169/ using document root <none> and password file <none>
I20260619 06:51:56.533202 29132 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:56.533408 29132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:56.533718 29132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:56.536641 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "58a77364e7af496c81096b7a40e5b4e2"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "cadea8e544191ae2e9f258b04304a62c"
server_key_iv: "b783ec06d16a515accdbc28f1cef423a"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.537379 29132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "58a77364e7af496c81096b7a40e5b4e2"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "cadea8e544191ae2e9f258b04304a62c"
server_key_iv: "b783ec06d16a515accdbc28f1cef423a"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.542520 29132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:51:56.546129 29147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:56.547436 29132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:51:56.547605 29132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "58a77364e7af496c81096b7a40e5b4e2"
format_stamp: "Formatted at 2026-06-19 06:51:56 on dist-test-slave-1h5s"
server_key: "cadea8e544191ae2e9f258b04304a62c"
server_key_iv: "b783ec06d16a515accdbc28f1cef423a"
server_key_version: "encryptionkey@0"
I20260619 06:51:56.547803 29132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:56.569941 29132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:56.574450 29132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:56.574762 29132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:56.575727 29132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:56.577086 29132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:56.577162 29132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:56.577353 29132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:56.577410 29132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 06:51:56.610741 29132 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:36629
I20260619 06:51:56.610777 29259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:36629 every 8 connection(s)
I20260619 06:51:56.612075 29132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:51:56.616418 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 29132
I20260619 06:51:56.616619 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:51:56.617185 18589 external_mini_cluster.cc:1428] Setting key e0f482cf6e3330c8c3d8729a692e8c06
Jun 19 06:51:56 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851916, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:56.635373 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:56.614657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:60667 (local address 127.18.39.126:46547)
0619 06:51:56.614831 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:56.614835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:56.615891 (+ 1056us) server_negotiation.cc:407] Connection header received
0619 06:51:56.618490 (+ 2599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:56.618496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:56.618582 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:56.618704 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:56.620672 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.621436 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.623878 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.624307 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.624407 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:56.629172 (+ 4765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:56.629178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:56.629181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:56.629219 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:56.631858 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:56.632921 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:56.632925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:56.632927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:56.633118 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:56.633655 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:56.633658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:56.633660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:56.633925 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:56.634049 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:56.634954 (+ 905us) server_negotiation.cc:299] Negotiation successful
0619 06:51:56.635134 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:51:56.636955 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:56.615068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46547 (local address 127.18.39.66:60667)
0619 06:51:56.615733 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:56.615769 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:56.618188 (+ 2419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:56.619006 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:56.619018 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:56.619665 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:56.620473 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:56.620502 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.621587 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.621594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:56.623710 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:56.623728 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:56.624552 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:56.624562 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:56.624922 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:56.625763 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:56.625803 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:56.628891 (+ 3088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:56.632037 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:56.632049 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:56.632072 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:56.632716 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:56.633246 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:56.633253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:56.633260 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:56.633542 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:56.634051 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:56.634063 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:56.634556 (+ 493us) client_negotiation.cc:769] Sending connection context
0619 06:51:56.634863 (+ 307us) client_negotiation.cc:240] Negotiation successful
0619 06:51:56.635460 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":254,"threads_started":1}
I20260619 06:51:56.638543 29260 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46547
I20260619 06:51:56.638895 29260 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:56.639843 29260 heartbeater.cc:511] Master 127.18.39.126:46547 requested a full tablet report, sending...
I20260619 06:51:56.641615 28937 ts_manager.cc:194] Registered new tserver with Master: 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629)
I20260619 06:51:56.642422 28937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:60667
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:51:56.670778 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:51:57.002282 29265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:51:57.002576 29265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:51:57.002776 29265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:51:57.010162 29265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:51:57.010308 29265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:51:57.010496 29265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:51:57.020546 29265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:46547
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:51:57.021951 29265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:51:57.023250 29265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:51:57.036003 29270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:57.035971 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:51:57.036059 29271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:51:57.036832 29265 server_base.cc:1034] running on GCE node
I20260619 06:51:57.037500 29265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:51:57.038439 29265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:51:57.039631 29265 hybrid_clock.cc:648] HybridClock initialized: now 1781851917039588 us; error 33 us; skew 500 ppm
I20260619 06:51:57.039945 29265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:51:57 dist-test-slave-1h5s krb5kdc[28892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851917, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:51:57.043242 29265 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:51:57.044873 29265 webserver.cc:460] Webserver started at http://127.18.39.67:45251/ using document root <none> and password file <none>
I20260619 06:51:57.045667 29265 fs_manager.cc:359] Metadata directory not provided
I20260619 06:51:57.045766 29265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:51:57.046103 29265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:51:57.049000 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "ac47e4a9c6c741ae8e08031001a1e8ae"
format_stamp: "Formatted at 2026-06-19 06:51:57 on dist-test-slave-1h5s"
server_key: "fe3950a63df6569c116cb77c833c6b95"
server_key_iv: "ff7f8c3169864d14552aa7ccc35f0208"
server_key_version: "encryptionkey@0"
I20260619 06:51:57.049755 29265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "ac47e4a9c6c741ae8e08031001a1e8ae"
format_stamp: "Formatted at 2026-06-19 06:51:57 on dist-test-slave-1h5s"
server_key: "fe3950a63df6569c116cb77c833c6b95"
server_key_iv: "ff7f8c3169864d14552aa7ccc35f0208"
server_key_version: "encryptionkey@0"
I20260619 06:51:57.054901 29265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:51:57.058302 29280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:57.059674 29265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:51:57.059810 29265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "ac47e4a9c6c741ae8e08031001a1e8ae"
format_stamp: "Formatted at 2026-06-19 06:51:57 on dist-test-slave-1h5s"
server_key: "fe3950a63df6569c116cb77c833c6b95"
server_key_iv: "ff7f8c3169864d14552aa7ccc35f0208"
server_key_version: "encryptionkey@0"
I20260619 06:51:57.059978 29265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:51:57.081223 29265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:51:57.085110 29265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:51:57.085381 29265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:51:57.086263 29265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:51:57.087489 29265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:51:57.087560 29265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:57.087639 29265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:51:57.087682 29265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:51:57.121284 29265 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:33619
I20260619 06:51:57.121381 29392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:33619 every 8 connection(s)
I20260619 06:51:57.122632 29265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:51:57.123564 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 29265
I20260619 06:51:57.123760 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:51:57.124302 18589 external_mini_cluster.cc:1428] Setting key d4137a8c17dc7cb63b469d56a91641bf
I20260619 06:51:57.126175 29126 heartbeater.cc:503] Master 127.18.39.126:46547 was elected leader, sending a full tablet report...
Jun 19 06:51:57 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851917, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:51:57.143265 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.125473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:45977 (local address 127.18.39.126:46547)
0619 06:51:57.125665 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.125672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.127037 (+ 1365us) server_negotiation.cc:407] Connection header received
0619 06:51:57.129240 (+ 2203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.129244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.129311 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.129470 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:57.131066 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.131881 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.133740 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.134158 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.134272 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.137731 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:57.137736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:57.137739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:57.137770 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:57.140478 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.141286 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.141290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.141292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.141496 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.141862 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.141865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.141867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.142114 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:57.142218 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.142867 (+ 649us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.143021 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:51:57.144315 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.125986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:46547 (local address 127.18.39.67:45977)
0619 06:51:57.126862 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.126927 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.128948 (+ 2021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.129669 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.129677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.130239 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:51:57.130835 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.130852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.132070 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.132074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.133571 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.133584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.134355 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.134360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.134572 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.135309 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:51:57.135333 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:51:57.137526 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:51:57.140602 (+ 3076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:57.140611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:57.140628 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:57.141172 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:57.141589 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:51:57.141593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:51:57.141595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:51:57.141769 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:51:57.142212 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:51:57.142217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:51:57.142545 (+ 328us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.142772 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.143161 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":341,"threads_started":1}
I20260619 06:51:57.145736 29393 heartbeater.cc:347] Connected to a master server at 127.18.39.126:46547
I20260619 06:51:57.146029 29393 heartbeater.cc:464] Registering TS with master...
I20260619 06:51:57.146734 29393 heartbeater.cc:511] Master 127.18.39.126:46547 requested a full tablet report, sending...
I20260619 06:51:57.148231 28936 ts_manager.cc:194] Registered new tserver with Master: ac47e4a9c6c741ae8e08031001a1e8ae (127.18.39.67:33619)
I20260619 06:51:57.149071 28936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:45977
I20260619 06:51:57.153242 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:51:57.169629 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.157627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.18.39.126:46547)
0619 06:51:57.157784 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.157788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.158440 (+ 652us) server_negotiation.cc:407] Connection header received
0619 06:51:57.158588 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.158591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.158651 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.158728 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:57.160085 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.160815 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.162056 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.162442 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.162533 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.164977 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:57.164982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:57.164985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:57.165016 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:57.167396 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.168010 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.168014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.168017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.168131 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.168454 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.168457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.168459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.168737 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:57.168908 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.169196 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.169402 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:51:57.173764 28936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:51:57.180294 28936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:51:57.214082 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.191944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.18.39.65:35361)
0619 06:51:57.205219 (+ 13275us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.205232 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.205271 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:51:57.205416 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.205438 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.205774 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.205986 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.206736 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.208789 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.211787 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.212812 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.212966 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.213203 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.213525 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.213706 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13052,"thread_start_us":12851,"threads_started":1}
I20260619 06:51:57.214874 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.191537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:35361 (local address 127.0.0.1:58588)
0619 06:51:57.203273 (+ 11736us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.203343 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.203609 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.206266 (+ 2657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.206271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.206295 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.206575 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.206583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.209001 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.209010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.211623 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.211638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.213009 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.213015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.213202 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.213265 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.214303 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.214489 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11372,"mutex_wait_us":27,"thread_start_us":13678,"threads_started":1}
I20260619 06:51:57.215516 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.192108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:33619 (local address 127.0.0.1:38878)
0619 06:51:57.205532 (+ 13424us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.205553 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.205669 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.207284 (+ 1615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.207292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.207327 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.207865 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.207879 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.210387 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.210392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.212637 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.212651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.214609 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.214616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.214787 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.214837 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.214916 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.215047 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13279,"mutex_wait_us":1,"thread_start_us":11758,"threads_started":1}
I20260619 06:51:57.216600 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.192367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.18.39.67:33619)
0619 06:51:57.206294 (+ 13927us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.206305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.206343 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:51:57.206449 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.206460 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.206719 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.206937 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.208110 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.210219 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.212831 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.214380 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.215853 (+ 1473us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.216083 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.216144 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.216295 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13704,"thread_start_us":13562,"threads_started":1}
I20260619 06:51:57.219147 29410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.206854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36629 (local address 127.0.0.1:39996)
0619 06:51:57.210506 (+ 3652us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.210529 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.210671 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.211338 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.211344 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.211368 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.211841 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.211852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.213926 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.213932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.216330 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.216344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.218425 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.218444 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.218601 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.218633 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.218727 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.218842 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3500,"thread_start_us":3348,"threads_started":1}
I20260619 06:51:57.219466 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.205915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.18.39.66:36629)
0619 06:51:57.210090 (+ 4175us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.210098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.210788 (+ 690us) server_negotiation.cc:407] Connection header received
0619 06:51:57.210884 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.210893 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.211114 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.212038 (+ 924us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.212249 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.213733 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.216525 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.217999 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.218128 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.218298 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.218795 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.218955 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4016,"thread_start_us":3907,"threads_started":1}
I20260619 06:51:57.219007 29061 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f (DEFAULT_TABLE table=test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:57.220376 29061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec20cb3ca7542a1975355eef402fd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:57.219940 29328 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f (DEFAULT_TABLE table=test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:57.221112 29328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec20cb3ca7542a1975355eef402fd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:57.222887 29195 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f (DEFAULT_TABLE table=test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:51:57.224145 29195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec20cb3ca7542a1975355eef402fd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:51:57.233800 29411 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Bootstrap starting.
I20260619 06:51:57.234668 29412 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Bootstrap starting.
I20260619 06:51:57.237461 29411 tablet_bootstrap.cc:654] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:57.237646 29413 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Bootstrap starting.
I20260619 06:51:57.238219 29412 tablet_bootstrap.cc:654] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:57.239058 29411 log.cc:824] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:57.239333 29412 log.cc:824] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:57.240567 29413 tablet_bootstrap.cc:654] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Neither blocks nor log segments found. Creating new log.
I20260619 06:51:57.241554 29413 log.cc:824] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:51:57.243772 29412 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: No bootstrap required, opened a new log
I20260619 06:51:57.244042 29412 ts_tablet_manager.cc:1397] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260619 06:51:57.244355 29413 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: No bootstrap required, opened a new log
I20260619 06:51:57.244575 29411 tablet_bootstrap.cc:492] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: No bootstrap required, opened a new log
I20260619 06:51:57.244603 29413 ts_tablet_manager.cc:1397] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260619 06:51:57.244818 29411 ts_tablet_manager.cc:1397] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260619 06:51:57.251739 29412 raft_consensus.cc:348] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.252149 29412 raft_consensus.cc:374] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:57.252254 29412 raft_consensus.cc:729] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa2258f13b2f4623a506c3a5d8f70269, State: Initialized, Role: FOLLOWER
I20260619 06:51:57.252936 29412 consensus_queue.cc:260] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.254743 29412 ts_tablet_manager.cc:1428] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260619 06:51:57.255228 29413 raft_consensus.cc:348] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.255416 29411 raft_consensus.cc:348] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.255702 29413 raft_consensus.cc:374] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:57.255810 29411 raft_consensus.cc:374] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:51:57.255841 29413 raft_consensus.cc:729] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a77364e7af496c81096b7a40e5b4e2, State: Initialized, Role: FOLLOWER
I20260619 06:51:57.255908 29411 raft_consensus.cc:729] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac47e4a9c6c741ae8e08031001a1e8ae, State: Initialized, Role: FOLLOWER
I20260619 06:51:57.256466 29411 consensus_queue.cc:260] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.256439 29413 consensus_queue.cc:260] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.258122 29260 heartbeater.cc:503] Master 127.18.39.126:46547 was elected leader, sending a full tablet report...
I20260619 06:51:57.258169 29411 ts_tablet_manager.cc:1428] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260619 06:51:57.258515 29413 ts_tablet_manager.cc:1428] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260619 06:51:57.259884 29393 heartbeater.cc:503] Master 127.18.39.126:46547 was elected leader, sending a full tablet report...
W20260619 06:51:57.345055 29127 tablet.cc:2369] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:57.364018 29261 tablet.cc:2369] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:51:57.374426 29394 tablet.cc:2369] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:51:57.420403 29417 raft_consensus.cc:482] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:57.420706 29417 raft_consensus.cc:504] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.422502 29417 leader_election.cc:288] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629), ac47e4a9c6c741ae8e08031001a1e8ae (127.18.39.67:33619)
I20260619 06:51:57.426178 29418 raft_consensus.cc:482] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:51:57.426527 29418 raft_consensus.cc:504] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.428596 29418 leader_election.cc:288] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa2258f13b2f4623a506c3a5d8f70269 (127.18.39.65:35361), 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629)
I20260619 06:51:57.430963 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.423044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:41615 (local address 127.18.39.66:36629)
0619 06:51:57.423282 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.423292 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.424057 (+ 765us) server_negotiation.cc:407] Connection header received
0619 06:51:57.424377 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.424384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.424474 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.424587 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.425247 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.426072 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.428092 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.429201 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.429389 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.429481 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.430522 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.430679 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:51:57.432041 29420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.422941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36629 (local address 127.18.39.65:41615)
0619 06:51:57.423888 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.423915 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.424142 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.424615 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.424619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.424683 (+ 64us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.425085 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.425094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.426211 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.426217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.427925 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.427937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.429633 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.429640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.429874 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.430357 (+ 483us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.431405 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.431570 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"mutex_wait_us":418,"spinlock_wait_cycles":21760,"thread_start_us":499,"threads_started":1}
I20260619 06:51:57.432351 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.422969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:60659 (local address 127.18.39.67:33619)
0619 06:51:57.423192 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.423199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.424711 (+ 1512us) server_negotiation.cc:407] Connection header received
0619 06:51:57.425000 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.425004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.425063 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.425155 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.425689 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.426904 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.429408 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.430741 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.430884 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.430961 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.431718 (+ 757us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.431860 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:51:57.432758 29421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.422986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:33619 (local address 127.18.39.65:60659)
0619 06:51:57.424611 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.424631 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.424841 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.425182 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.425186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.425210 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.425523 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.425531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.427060 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.427065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.429212 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.429228 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.431311 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.431318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.431537 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.431578 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.432227 (+ 649us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.432386 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1482,"mutex_wait_us":3,"thread_start_us":1250,"threads_started":1}
I20260619 06:51:57.433686 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "fa2258f13b2f4623a506c3a5d8f70269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" is_pre_election: true
I20260619 06:51:57.434104 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "fa2258f13b2f4623a506c3a5d8f70269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a77364e7af496c81096b7a40e5b4e2" is_pre_election: true
I20260619 06:51:57.434235 29348 raft_consensus.cc:2454] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa2258f13b2f4623a506c3a5d8f70269 in term 0.
I20260619 06:51:57.434652 29215 raft_consensus.cc:2454] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa2258f13b2f4623a506c3a5d8f70269 in term 0.
I20260619 06:51:57.434998 29014 leader_election.cc:302] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac47e4a9c6c741ae8e08031001a1e8ae, fa2258f13b2f4623a506c3a5d8f70269; no voters:
I20260619 06:51:57.435717 29417 raft_consensus.cc:2781] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:57.435878 29417 raft_consensus.cc:482] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:57.436007 29417 raft_consensus.cc:3037] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:57.436355 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.428958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:35361 (local address 127.18.39.67:57071)
0619 06:51:57.429374 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.429393 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.429510 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.430291 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.430298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.430330 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.430576 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.430586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.431998 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.432007 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.434042 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.434057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.435605 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.435615 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.435807 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.435841 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.435940 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.436088 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268}
I20260619 06:51:57.436470 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.429006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:57071 (local address 127.18.39.65:35361)
0619 06:51:57.429191 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.429197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.429623 (+ 426us) server_negotiation.cc:407] Connection header received
0619 06:51:57.429680 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.429685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.429756 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.429833 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.430757 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.431831 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.434221 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.435445 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.435603 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.435671 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.436074 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.436201 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":52}
I20260619 06:51:57.437598 29081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa2258f13b2f4623a506c3a5d8f70269" is_pre_election: true
I20260619 06:51:57.438098 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.430072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60595 (local address 127.18.39.66:36629)
0619 06:51:57.431244 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.431255 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.431273 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:57.431350 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.431358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.431428 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.431512 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:51:57.432230 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.433319 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.436155 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.437106 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.437275 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.437348 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.437690 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.437842 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1051,"thread_start_us":2960,"threads_started":1}
I20260619 06:51:57.438189 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.429322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36629 (local address 127.18.39.67:60595)
0619 06:51:57.430386 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0619 06:51:57.430404 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:51:57.430855 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:51:57.431642 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:51:57.431647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:51:57.431674 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:51:57.432034 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.432047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.433528 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.433536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.435997 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:51:57.436008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.437307 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.437312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:51:57.437511 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.437547 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:51:57.437642 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 06:51:57.437802 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"thread_start_us":747,"threads_started":1}
I20260619 06:51:57.438860 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a77364e7af496c81096b7a40e5b4e2" is_pre_election: true
I20260619 06:51:57.439073 29215 raft_consensus.cc:2454] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac47e4a9c6c741ae8e08031001a1e8ae in term 0.
I20260619 06:51:57.439281 29417 raft_consensus.cc:504] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.439597 29284 leader_election.cc:302] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a77364e7af496c81096b7a40e5b4e2, ac47e4a9c6c741ae8e08031001a1e8ae; no voters:
I20260619 06:51:57.439771 29081 raft_consensus.cc:2379] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ac47e4a9c6c741ae8e08031001a1e8ae in current term 1: Already voted for candidate fa2258f13b2f4623a506c3a5d8f70269 in this term.
I20260619 06:51:57.440062 29418 raft_consensus.cc:2781] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:51:57.440207 29418 raft_consensus.cc:482] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:51:57.440313 29418 raft_consensus.cc:3037] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:57.441851 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "fa2258f13b2f4623a506c3a5d8f70269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a77364e7af496c81096b7a40e5b4e2"
I20260619 06:51:57.441881 29348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "fa2258f13b2f4623a506c3a5d8f70269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae"
I20260619 06:51:57.442063 29215 raft_consensus.cc:3037] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:51:57.442236 29417 leader_election.cc:288] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [CANDIDATE]: Term 1 election: Requested vote from peers 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629), ac47e4a9c6c741ae8e08031001a1e8ae (127.18.39.67:33619)
I20260619 06:51:57.443157 29418 raft_consensus.cc:504] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.443609 29348 raft_consensus.cc:2379] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fa2258f13b2f4623a506c3a5d8f70269 in current term 1: Already voted for candidate ac47e4a9c6c741ae8e08031001a1e8ae in this term.
I20260619 06:51:57.443887 29418 leader_election.cc:288] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [CANDIDATE]: Term 1 election: Requested vote from peers fa2258f13b2f4623a506c3a5d8f70269 (127.18.39.65:35361), 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629)
I20260619 06:51:57.444459 29081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa2258f13b2f4623a506c3a5d8f70269"
I20260619 06:51:57.444698 29081 raft_consensus.cc:2379] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ac47e4a9c6c741ae8e08031001a1e8ae in current term 1: Already voted for candidate fa2258f13b2f4623a506c3a5d8f70269 in this term.
I20260619 06:51:57.444684 29215 raft_consensus.cc:2454] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa2258f13b2f4623a506c3a5d8f70269 in term 1.
I20260619 06:51:57.445340 29017 leader_election.cc:302] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 58a77364e7af496c81096b7a40e5b4e2, fa2258f13b2f4623a506c3a5d8f70269; no voters: ac47e4a9c6c741ae8e08031001a1e8ae
I20260619 06:51:57.445588 29215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec20cb3ca7542a1975355eef402fd4f" candidate_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a77364e7af496c81096b7a40e5b4e2"
I20260619 06:51:57.445703 29417 raft_consensus.cc:2781] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:51:57.445923 29215 raft_consensus.cc:2379] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ac47e4a9c6c741ae8e08031001a1e8ae in current term 1: Already voted for candidate fa2258f13b2f4623a506c3a5d8f70269 in this term.
I20260619 06:51:57.446286 29417 raft_consensus.cc:686] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 LEADER]: Becoming Leader. State: Replica: fa2258f13b2f4623a506c3a5d8f70269, State: Running, Role: LEADER
I20260619 06:51:57.446491 29284 leader_election.cc:302] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac47e4a9c6c741ae8e08031001a1e8ae; no voters: 58a77364e7af496c81096b7a40e5b4e2, fa2258f13b2f4623a506c3a5d8f70269
I20260619 06:51:57.446867 29418 raft_consensus.cc:2726] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 06:51:57.446867 29417 consensus_queue.cc:237] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } }
I20260619 06:51:57.451092 28937 catalog_manager.cc:5495] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa2258f13b2f4623a506c3a5d8f70269 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "fa2258f13b2f4623a506c3a5d8f70269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa2258f13b2f4623a506c3a5d8f70269" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 35361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 } health_report { overall_health: UNKNOWN } } }
I20260619 06:51:57.514984 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.509450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.18.39.65:35361)
0619 06:51:57.509698 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.509705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.509816 (+ 111us) server_negotiation.cc:407] Connection header received
0619 06:51:57.510082 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.510087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.510161 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.510326 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:51:57.510736 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.511512 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.512810 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.513229 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.513381 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.513863 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:51:57.514371 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:51:57.514529 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.514605 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.514685 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 06:51:57.523418 29215 raft_consensus.cc:1261] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 1 FOLLOWER]: Refusing update from remote peer fa2258f13b2f4623a506c3a5d8f70269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:57.523438 29348 raft_consensus.cc:1261] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Refusing update from remote peer fa2258f13b2f4623a506c3a5d8f70269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:51:57.524262 29424 consensus_queue.cc:1035] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a77364e7af496c81096b7a40e5b4e2" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:57.524643 29417 consensus_queue.cc:1035] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac47e4a9c6c741ae8e08031001a1e8ae" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:51:57.541894 29428 mvcc.cc:205] Tried to move back new op lower bound from 7298465454164512768 to 7298465453865811968. Current Snapshot: MvccSnapshot[applied={T|T < 7298465454164512768}]
I20260619 06:51:57.553294 28936 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:57.553660 28936 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:51:57.557385 28936 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d934839ec87c4c5bb34a913625c04cfe: Sending DeleteTablet for 3 replicas of tablet 7ec20cb3ca7542a1975355eef402fd4f
I20260619 06:51:57.558615 29328 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38878
I20260619 06:51:57.558723 29195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:39996
I20260619 06:51:57.558909 29061 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec20cb3ca7542a1975355eef402fd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:51:57 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:58588
I20260619 06:51:57.560217 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28998
I20260619 06:51:57.574594 29439 tablet_replica.cc:330] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: stopping tablet replica
I20260619 06:51:57.575122 29439 raft_consensus.cc:2229] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:51:57.575374 29437 tablet_replica.cc:330] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: stopping tablet replica
I20260619 06:51:57.575814 29439 raft_consensus.cc:2258] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:57.575850 29437 raft_consensus.cc:2229] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:57.576411 29437 raft_consensus.cc:2258] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:57.577806 29439 ts_tablet_manager.cc:1905] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:57.579630 29438 tablet_replica.cc:330] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: stopping tablet replica
I20260619 06:51:57.580026 29438 raft_consensus.cc:2229] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:51:57.580489 29438 raft_consensus.cc:2258] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:51:57.583338 29438 ts_tablet_manager.cc:1905] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:57.583746 29439 ts_tablet_manager.cc:1918] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:57.583930 29439 log.cc:1196] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/7ec20cb3ca7542a1975355eef402fd4f
I20260619 06:51:57.584399 29439 ts_tablet_manager.cc:1939] T 7ec20cb3ca7542a1975355eef402fd4f P fa2258f13b2f4623a506c3a5d8f70269: Deleting consensus metadata
I20260619 06:51:57.586061 28924 catalog_manager.cc:4841] TS fa2258f13b2f4623a506c3a5d8f70269 (127.18.39.65:35361): tablet 7ec20cb3ca7542a1975355eef402fd4f (table test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]) successfully deleted
I20260619 06:51:57.587450 29437 ts_tablet_manager.cc:1905] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:51:57.590369 29438 ts_tablet_manager.cc:1918] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:57.590528 29438 log.cc:1196] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/7ec20cb3ca7542a1975355eef402fd4f
I20260619 06:51:57.591058 29438 ts_tablet_manager.cc:1939] T 7ec20cb3ca7542a1975355eef402fd4f P 58a77364e7af496c81096b7a40e5b4e2: Deleting consensus metadata
Jun 19 06:51:57 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:51:57.592921 28924 catalog_manager.cc:4841] TS 58a77364e7af496c81096b7a40e5b4e2 (127.18.39.66:36629): tablet 7ec20cb3ca7542a1975355eef402fd4f (table test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]) successfully deleted
I20260619 06:51:57.593822 29437 ts_tablet_manager.cc:1918] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:51:57.594012 29437 log.cc:1196] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/7ec20cb3ca7542a1975355eef402fd4f
I20260619 06:51:57.594455 29437 ts_tablet_manager.cc:1939] T 7ec20cb3ca7542a1975355eef402fd4f P ac47e4a9c6c741ae8e08031001a1e8ae: Deleting consensus metadata
I20260619 06:51:57.596554 28921 catalog_manager.cc:4841] TS ac47e4a9c6c741ae8e08031001a1e8ae (127.18.39.67:33619): tablet 7ec20cb3ca7542a1975355eef402fd4f (table test-table [id=8adcb41c4c2f4a3d93db1eac5509f4cc]) successfully deleted
I20260619 06:51:57.601243 29406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.561506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59810 (local address 127.18.39.65:35361)
0619 06:51:57.561750 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.561759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.572101 (+ 10342us) server_negotiation.cc:407] Connection header received
0619 06:51:57.572568 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.572575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.572660 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.572746 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:57.574904 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.576059 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.579137 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.579890 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.580066 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.593755 (+ 13689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:57.593770 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:57.593790 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:57.593849 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:57.598072 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.598829 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.598840 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.598844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.599028 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.599500 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.599505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.599509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.600019 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:57.600351 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.600653 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.600881 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:51:57.602080 29121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:57.702205 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28998
I20260619 06:51:57.720752 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 29132
Jun 19 06:51:57 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:51:57.735605 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.721694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.18.39.66:36629)
0619 06:51:57.721885 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.721890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.721908 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:51:57.721963 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.721967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.722046 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.722130 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:57.723344 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.724105 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.725309 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.725813 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.725914 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.730035 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:57.730052 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:57.730069 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:57.730106 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:57.733002 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.733820 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.733827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.733829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.733952 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.734263 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.734266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.734268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.734618 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:57.734832 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.735078 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.735240 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:51:57.736291 29255 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:57.827960 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 29132
I20260619 06:51:57.844930 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 29265
Jun 19 06:51:57 dist-test-slave-1h5s krb5kdc[28892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:51:57.860458 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:51:57.845781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.18.39.67:33619)
0619 06:51:57.846072 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 06:51:57.846080 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:51:57.846139 (+ 59us) server_negotiation.cc:407] Connection header received
0619 06:51:57.846362 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:51:57.846368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:51:57.846451 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:51:57.846572 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:51:57.847853 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.848592 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.849942 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:51:57.850340 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:51:57.850471 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:51:57.854764 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:51:57.854780 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:51:57.854798 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:51:57.854832 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:51:57.857682 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.858353 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.858361 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.858363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.858489 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:51:57.858855 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:51:57.858862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:51:57.858865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:51:57.859250 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:51:57.859558 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:51:57.859872 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 06:51:57.860098 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260619 06:51:57.861112 29388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:57.954157 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 29265
I20260619 06:51:57.971232 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 28905
I20260619 06:51:57.972441 28967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:51:58.088202 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 28905
2026-06-19T06:51:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4418 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:51:58 dist-test-slave-1h5s krb5kdc[29447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:51:58 dist-test-slave-1h5s krb5kdc[29447](info): set up 2 sockets
Jun 19 06:51:58 dist-test-slave-1h5s krb5kdc[29447](info): commencing operation
krb5kdc: starting...
W20260619 06:51:59.194264 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:51:59 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:51:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangeradmin_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangeradmin_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangerlookup_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangerlookup_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
I20260619 06:51:59.518544 18589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHLTYaR/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/postgres -l logfile start
2026-06-19 06:52:02.017 UTC [29477] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 06:52:02.017 UTC [29477] LOG: listening on IPv4 address "127.18.39.84", port 33935
2026-06-19 06:52:02.021 UTC [29477] LOG: listening on Unix socket "/tmp/.s.PGSQL.33935"
2026-06-19 06:52:02.045 UTC [29480] LOG: database system was shut down at 2026-06-19 06:52:00 UTC
2026-06-19 06:52:02.049 UTC [29477] LOG: database system is ready to accept connections
I20260619 06:52:03.030002 18589 mini_postgres.cc:96] Postgres bound to 33935
2026-06-19 06:52:03.048 UTC [29488] FATAL: database "slave" does not exist
127.18.39.84:33935 - accepting connections
I20260619 06:52:03.048998 18589 mini_ranger.cc:161] Starting Ranger...
I20260619 06:52:03.076522 18589 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 06:52:03.561089 18589 mini_ranger.cc:87] Created ranger Postgres database
I20260619 06:52:03.561231 18589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin
2026-06-19 06:52:03,847 [I] DB FLAVOR :POSTGRES
2026-06-19 06:52:03,847 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:52:03,847 [I] Checking connection..
2026-06-19 06:52:03,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 06:52:04,116 [I] Checking connection passed.
2026-06-19 06:52:04,116 [I] --------- Verifying version history table ---------
2026-06-19 06:52:04,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:04,366 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:52:04,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:04,620 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:52:04,620 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 06:52:04,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 06:52:04.870 UTC [29552] WARNING: there is no transaction in progress
2026-06-19 06:52:04,876 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 06:52:04,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:05,146 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 06:52:05,146 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:52:05,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 06:52:05,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 06:52:05,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1h5s.c.gcp-upstream.internal','N') ;"
2026-06-19 06:52:05,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:52:06,088 [I] Table x_portal_user does not exist in database ranger
2026-06-19 06:52:06,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:52:06,362 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 06:52:06,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:52:06,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:52:06,855 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 06:52:06,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 06:52:07.834 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.851 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.868 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.883 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.899 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.919 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.974 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.980 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:07.995 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.005 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.016 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.030 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.040 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.047 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.054 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08.061 UTC [29660] WARNING: there is no transaction in progress
2026-06-19 06:52:08,670 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 06:52:08,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 06:52:08,899 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 06:52:08,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:52:09,133 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 06:52:09,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:52:09,365 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 06:52:09,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:52:09,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:52:09,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1h5s.c.gcp-upstream.internal';"
2026-06-19 06:52:10,104 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 06:52:10,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 06:52:10,346 [I] DB_PATCHES have already been applied
I20260619 06:52:10.352125 18589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHLTYaR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 06:52:10.352224 18589 mini_ranger.cc:193] Using host: 127.18.39.84
I20260619 06:52:10.369417 18589 mini_ranger.cc:227] Ranger admin URL: http://127.18.39.84:41855
Jun 19, 2026 6:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:52:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:52:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 06:52:11.520668 18589 mini_ranger.cc:160] Time spent starting Ranger: real 8.472s user 0.006s sys 0.035s
Jun 19, 2026 6:52:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:52:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 6:52:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:52:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:52:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 06:52:32.958570 18589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangerkms_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/rangerkms_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.39.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.39.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 06:52:33 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851953, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 06:52:33.160470 18589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 06:52:33.181454 18589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 06:52:34.044251 18589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 06:52:34.044480 18589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms
I20260619 06:52:34.044667 18589 mini_ranger_kms.cc:227] Using postgres at 127.18.39.84:33935
2026-06-19 06:52:34,463 [I] DB FLAVOR :POSTGRES
2026-06-19 06:52:34,464 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:52:34,464 [I] Checking connection
2026-06-19 06:52:34,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 06:52:34,730 [I] connection success
2026-06-19 06:52:34,731 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 06:52:34,731 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 06:52:34,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 06:52:34,997 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 06:52:34,998 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:52:34,998 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 06:52:34,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:33935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 06:52:35,285 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 06:52:35.796515 18589 mini_ranger_kms.cc:310] Created kms service
I20260619 06:52:36.003199 18589 mini_ranger_kms.cc:326] Created kudu user
I20260619 06:52:36.051198 18589 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 06:52:36 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851953, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:52:36.487089 18589 mini_ranger_kms.cc:384] Added ranger policy
I20260619 06:52:36.487430 18589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 06:52:36.487504 18589 mini_ranger_kms.cc:240] Using host: 127.18.39.84
I20260619 06:52:36.508472 18589 mini_ranger_kms.cc:276] Ranger KMS PID: 29840
I20260619 06:52:36.508700 18589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.39.84:35363
Jun 19, 2026 6:52:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:52:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:52:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:52:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:52:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:52:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 06:52:38.833174 18589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.673s user 0.105s sys 0.335s
I20260619 06:52:38.833453 18589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 06:52:38.833671 18589 mini_ranger_kms.cc:401] 127.18.39.84:35363/kms/v1/keys
Jun 19 06:52:38 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851958, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 06:52:39 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851958, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:52:39 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851958, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:52:42 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:52:42.143307 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37967
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:37967
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:35363/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:52:42.499811 29891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:52:42.500175 29891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:52:42.500329 29891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:52:42.508177 29891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:52:42.508355 29891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:52:42.508486 29891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:52:42.508584 29891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:52:42.508677 29891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:52:42.519178 29891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.39.84:35363/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:37967
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:52:42.520581 29891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:52:42.522053 29891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:52:42.534669 29896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:42.534616 29899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:42.534592 29897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:52:42.535511 29891 server_base.cc:1034] running on GCE node
I20260619 06:52:42.536316 29891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:52:42.537767 29891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:52:42.538954 29891 hybrid_clock.cc:648] HybridClock initialized: now 1781851962538911 us; error 37 us; skew 500 ppm
I20260619 06:52:42.539266 29891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:52:42 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:52:42.545238 29891 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:52:42.546878 29891 webserver.cc:460] Webserver started at http://127.18.39.126:43309/ using document root <none> and password file <none>
I20260619 06:52:42.547827 29891 fs_manager.cc:359] Metadata directory not provided
I20260619 06:52:42.547987 29891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:52:42.548341 29891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:52:42 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:52:42.606158 29891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "70353d71b8084f369cbaae51eb93f650"
format_stamp: "Formatted at 2026-06-19 06:52:42 on dist-test-slave-1h5s"
server_key: "01825775f0f0c2796fb12c279e173e10"
server_key_iv: "72833e3f183edac6b14f6a292f6b2cae"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:42.607183 29891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "70353d71b8084f369cbaae51eb93f650"
format_stamp: "Formatted at 2026-06-19 06:52:42 on dist-test-slave-1h5s"
server_key: "01825775f0f0c2796fb12c279e173e10"
server_key_iv: "72833e3f183edac6b14f6a292f6b2cae"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:42.613246 29891 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260619 06:52:42.633710 29906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:42.635581 29891 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260619 06:52:42.635784 29891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "70353d71b8084f369cbaae51eb93f650"
format_stamp: "Formatted at 2026-06-19 06:52:42 on dist-test-slave-1h5s"
server_key: "01825775f0f0c2796fb12c279e173e10"
server_key_iv: "72833e3f183edac6b14f6a292f6b2cae"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:42.636062 29891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:52:42.663830 29891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:52:42.668365 29891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:52:42.668684 29891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:52:42.693395 29891 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:37967
I20260619 06:52:42.693403 29957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:37967 every 8 connection(s)
I20260619 06:52:42.694823 29891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:52:42.695312 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 29891
I20260619 06:52:42.695519 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:52:42.696435 18589 external_mini_cluster.cc:1428] Setting key 2ba87d5fdadae853459b060db43d143a
I20260619 06:52:42.700582 29958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:52:42.714416 29958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: Bootstrap starting.
I20260619 06:52:42.718107 29958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: Neither blocks nor log segments found. Creating new log.
I20260619 06:52:42.719465 29958 log.cc:824] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:52:42 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:52:42.722784 29958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: No bootstrap required, opened a new log
I20260619 06:52:42.733242 29961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:42.699223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.18.39.126:37967)
0619 06:52:42.705988 (+ 6765us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:42.706005 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:42.707088 (+ 1083us) server_negotiation.cc:407] Connection header received
0619 06:52:42.709921 (+ 2833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:42.709956 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:42.710624 (+ 668us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:42.711151 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:42.712738 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:42.715093 (+ 2355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:42.716953 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:42.718162 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:42.718376 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:42.723415 (+ 5039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:42.723427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:42.723451 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:42.723497 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:42.728246 (+ 4749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:42.729249 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:42.729263 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:42.729273 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:42.729551 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:42.730013 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:42.730019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:42.730023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:42.730861 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:42.731195 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:42.731643 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 06:52:42.732093 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6474,"thread_start_us":6241,"threads_started":1}
I20260619 06:52:42.734009 29958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } }
I20260619 06:52:42.734449 29958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:52:42.734558 29958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70353d71b8084f369cbaae51eb93f650, State: Initialized, Role: FOLLOWER
I20260619 06:52:42.735164 29958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } }
I20260619 06:52:42.735381 29958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:52:42.735543 29958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:52:42.735733 29958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:52:42.738246 29958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } }
I20260619 06:52:42.738763 29958 leader_election.cc:302] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70353d71b8084f369cbaae51eb93f650; no voters:
I20260619 06:52:42.739729 29958 leader_election.cc:288] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:52:42.739884 29963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:52:42.741647 29963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [term 1 LEADER]: Becoming Leader. State: Replica: 70353d71b8084f369cbaae51eb93f650, State: Running, Role: LEADER
I20260619 06:52:42.742014 29958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:52:42.742233 29963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } }
I20260619 06:52:42.745261 29965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70353d71b8084f369cbaae51eb93f650. Latest consensus state: current_term: 1 leader_uuid: "70353d71b8084f369cbaae51eb93f650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } } }
I20260619 06:52:42.745239 29964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70353d71b8084f369cbaae51eb93f650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70353d71b8084f369cbaae51eb93f650" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37967 } } }
I20260619 06:52:42.745705 29964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [sys.catalog]: This master's current role is: LEADER
I20260619 06:52:42.745705 29965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650 [sys.catalog]: This master's current role is: LEADER
I20260619 06:52:42.746497 29972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:52:42.753906 29972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:52:42.754650 29958 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 06:52:42.784765 29972 catalog_manager.cc:1347] Generated new cluster ID: 875f912ab29a476583da504af670c16c
I20260619 06:52:42.784977 29972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:52:42.793918 29972 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:52:42.795306 29972 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:52:42.815316 29972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: Generated new TSK 0
I20260619 06:52:42.816733 29972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:52:43.589600 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:35363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:52:43 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 06:52:43.986557 30003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:52:43.986892 30003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:52:43.987071 30003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:52:43.995620 30003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:52:43.995824 30003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:52:43.996026 30003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:52:44.008013 30003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.39.84:35363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:52:44.009522 30003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:52:44.010841 30003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:52:44.028492 30013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:44.028437 30014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:44.029178 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:52:44.029518 30003 server_base.cc:1034] running on GCE node
I20260619 06:52:44.030221 30003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:52:44.031385 30003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:52:44.032697 30003 hybrid_clock.cc:648] HybridClock initialized: now 1781851964032582 us; error 81 us; skew 500 ppm
I20260619 06:52:44.033056 30003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:52:44.036578 30003 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:52:44.038595 30003 webserver.cc:460] Webserver started at http://127.18.39.65:42983/ using document root <none> and password file <none>
I20260619 06:52:44.039628 30003 fs_manager.cc:359] Metadata directory not provided
I20260619 06:52:44.039773 30003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:52:44.040189 30003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:52:44.059211 30003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "127f276df5c8416496a000d9a30e7962"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "60376b859e6abb38ec19f1364c04ae1a"
server_key_iv: "db4eab187179ffad25983a3b1b215e88"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.060509 30003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "127f276df5c8416496a000d9a30e7962"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "60376b859e6abb38ec19f1364c04ae1a"
server_key_iv: "db4eab187179ffad25983a3b1b215e88"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.075901 30003 fs_manager.cc:691] Time spent creating directory manager: real 0.015s user 0.010s sys 0.003s
I20260619 06:52:44.108726 30023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:44.110638 30003 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.006s sys 0.008s
I20260619 06:52:44.110867 30003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "127f276df5c8416496a000d9a30e7962"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "60376b859e6abb38ec19f1364c04ae1a"
server_key_iv: "db4eab187179ffad25983a3b1b215e88"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.111116 30003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:52:44.141297 30003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:52:44.146700 30003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:52:44.147538 30003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:52:44.154817 30003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:52:44.156652 30003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:52:44.156839 30003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:44.156985 30003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:52:44.157071 30003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:44.207721 30003 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:33311
I20260619 06:52:44.207826 30135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:33311 every 8 connection(s)
I20260619 06:52:44.209084 30003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:52:44.209527 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 30003
I20260619 06:52:44.209765 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:52:44.210758 18589 external_mini_cluster.cc:1428] Setting key 4a1d41afb4409112c633db1c662e8430
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:52:44.270380 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:44.212466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:36377 (local address 127.18.39.126:37967)
0619 06:52:44.218902 (+ 6436us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:44.218912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:44.227074 (+ 8162us) server_negotiation.cc:407] Connection header received
0619 06:52:44.241999 (+ 14925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:44.242005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:44.242134 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:44.242274 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:44.247609 (+ 5335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:44.248814 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:44.251437 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:44.251936 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:44.252144 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:44.257568 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:44.257580 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:44.257587 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:44.257649 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:44.261647 (+ 3998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:44.262779 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:44.262788 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:44.262794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:44.262974 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:44.267838 (+ 4864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:44.267844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:44.267850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:44.268272 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:44.268426 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:44.269582 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0619 06:52:44.269861 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5124,"thread_start_us":4985,"threads_started":1}
I20260619 06:52:44.273021 30139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:44.213128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:37967 (local address 127.18.39.65:36377)
0619 06:52:44.226562 (+ 13434us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:44.226653 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:44.240905 (+ 14252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:44.245761 (+ 4856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:44.245776 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:44.246455 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:52:44.247364 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:44.247392 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:44.249053 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:44.249059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:44.251248 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:44.251261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:44.252256 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:44.252264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:44.252652 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:44.253579 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:52:44.253613 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:52:44.257184 (+ 3571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:52:44.261890 (+ 4706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:44.261902 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:44.261926 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:44.262607 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:44.263143 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:44.263148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:44.263151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:44.263402 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:44.268530 (+ 5128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:52:44.268542 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:52:44.269046 (+ 504us) client_negotiation.cc:769] Sending connection context
0619 06:52:44.270873 (+ 1827us) client_negotiation.cc:240] Negotiation successful
0619 06:52:44.271510 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12959,"thread_start_us":12667,"threads_started":1}
I20260619 06:52:44.275064 30136 heartbeater.cc:347] Connected to a master server at 127.18.39.126:37967
I20260619 06:52:44.275463 30136 heartbeater.cc:464] Registering TS with master...
I20260619 06:52:44.276547 30136 heartbeater.cc:511] Master 127.18.39.126:37967 requested a full tablet report, sending...
I20260619 06:52:44.279186 29923 ts_manager.cc:194] Registered new tserver with Master: 127f276df5c8416496a000d9a30e7962 (127.18.39.65:33311)
I20260619 06:52:44.281306 29923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:36377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:52:44.290213 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:35363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851963, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
W20260619 06:52:44.739243 30142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:52:44.739755 30142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:52:44.739961 30142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:52:44.751082 30142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:52:44.751247 30142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:52:44.751461 30142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:52:44.766619 30142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.39.84:35363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:52:44.768683 30142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:52:44.770525 30142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:52:44.806193 30149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:44.807425 30150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:52:44.817955 30142 server_base.cc:1034] running on GCE node
W20260619 06:52:44.818766 30152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:52:44.819694 30142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:52:44.823814 30142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:52:44.840981 30142 hybrid_clock.cc:648] HybridClock initialized: now 1781851964837743 us; error 3186 us; skew 500 ppm
I20260619 06:52:44.842859 30142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:52:44.851109 30142 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:52:44.902458 30142 webserver.cc:460] Webserver started at http://127.18.39.66:46121/ using document root <none> and password file <none>
I20260619 06:52:44.903595 30142 fs_manager.cc:359] Metadata directory not provided
I20260619 06:52:44.903827 30142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:52:44.904256 30142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:52:44 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:52:44.924141 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "c4d7d019daaf4b06be5f47842d0a2072"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "80b1790a12f2c08a160a9e7d4ac33f18"
server_key_iv: "957c442b4a8c15ef4f1b590b9dd5c3c3"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.926373 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "c4d7d019daaf4b06be5f47842d0a2072"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "80b1790a12f2c08a160a9e7d4ac33f18"
server_key_iv: "957c442b4a8c15ef4f1b590b9dd5c3c3"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.947305 30142 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.009s sys 0.008s
I20260619 06:52:44.982156 30159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:44.984503 30142 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.008s sys 0.000s
I20260619 06:52:44.984792 30142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "c4d7d019daaf4b06be5f47842d0a2072"
format_stamp: "Formatted at 2026-06-19 06:52:44 on dist-test-slave-1h5s"
server_key: "80b1790a12f2c08a160a9e7d4ac33f18"
server_key_iv: "957c442b4a8c15ef4f1b590b9dd5c3c3"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:44.985183 30142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:52:45.041245 30142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:52:45.047072 30142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:52:45.047470 30142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:52:45.049777 30142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:52:45.051548 30142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:52:45.051641 30142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:45.052012 30142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:52:45.052115 30142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:45.120428 30142 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:42473
I20260619 06:52:45.120464 30278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:42473 every 8 connection(s)
I20260619 06:52:45.122141 30142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:52:45.126307 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 30142
I20260619 06:52:45.126507 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:52:45.127130 18589 external_mini_cluster.cc:1428] Setting key aa9b532038d8eaa03c20b45760e91532
Jun 19 06:52:45 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851964, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:52:45.148347 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.125399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:32905 (local address 127.18.39.126:37967)
0619 06:52:45.125899 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.125904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.127170 (+ 1266us) server_negotiation.cc:407] Connection header received
0619 06:52:45.129224 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.129229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.129316 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.129454 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:45.131482 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.132691 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.135036 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.135762 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.135925 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.140342 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:45.140349 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:45.140353 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:45.140392 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:45.144270 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.145383 (+ 1113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.145390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.145394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.145538 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.146119 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.146124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.146128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.146501 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:45.146637 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.147706 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.147961 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
I20260619 06:52:45.149964 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.126033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:37967 (local address 127.18.39.66:32905)
0619 06:52:45.126985 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:45.127037 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:45.128937 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:45.129737 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:45.129749 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:45.130428 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:52:45.131254 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.131278 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.132883 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.132889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.134882 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.134896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.136019 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.136027 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.136310 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.137121 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:52:45.137150 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:52:45.140078 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:52:45.144447 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:45.144461 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:45.144485 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:45.145175 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:45.145669 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:45.145677 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:45.145683 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:45.145965 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:45.146669 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:52:45.146681 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:52:45.147208 (+ 527us) client_negotiation.cc:769] Sending connection context
0619 06:52:45.147570 (+ 362us) client_negotiation.cc:240] Negotiation successful
0619 06:52:45.148238 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":399,"threads_started":1}
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:52:45.151985 30279 heartbeater.cc:347] Connected to a master server at 127.18.39.126:37967
I20260619 06:52:45.152323 30279 heartbeater.cc:464] Registering TS with master...
I20260619 06:52:45.153067 30279 heartbeater.cc:511] Master 127.18.39.126:37967 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:52:45.154786 29923 ts_manager.cc:194] Registered new tserver with Master: c4d7d019daaf4b06be5f47842d0a2072 (127.18.39.66:42473)
I20260619 06:52:45.155946 29923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:32905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:52:45.181173 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:35363/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:52:45.285215 30136 heartbeater.cc:503] Master 127.18.39.126:37967 was elected leader, sending a full tablet report...
W20260619 06:52:45.538255 30285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:52:45.538597 30285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:52:45.538763 30285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:52:45.546458 30285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:52:45.546622 30285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:52:45.546793 30285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:52:45.557487 30285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.39.84:35363/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:37967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:52:45.558948 30285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:52:45.560422 30285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:52:45.574748 30291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:45.574769 30293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:52:45.575364 30290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:52:45.575959 30285 server_base.cc:1034] running on GCE node
I20260619 06:52:45.576638 30285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:52:45.577926 30285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:52:45.579190 30285 hybrid_clock.cc:648] HybridClock initialized: now 1781851965579106 us; error 87 us; skew 500 ppm
I20260619 06:52:45.579540 30285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:52:45 dist-test-slave-1h5s krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851965, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:52:45.583628 30285 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:52:45.585196 30285 webserver.cc:460] Webserver started at http://127.18.39.67:40009/ using document root <none> and password file <none>
I20260619 06:52:45.585947 30285 fs_manager.cc:359] Metadata directory not provided
I20260619 06:52:45.586089 30285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:52:45.586400 30285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:52:45 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851965, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:52:45.605175 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "41012378b41442e7b1a3ec533e5ea053"
format_stamp: "Formatted at 2026-06-19 06:52:45 on dist-test-slave-1h5s"
server_key: "230208406a26d3f111a20bebabe7de9c"
server_key_iv: "820b46aa5bff086dde26b48bf49d5fe1"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:45.606365 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "41012378b41442e7b1a3ec533e5ea053"
format_stamp: "Formatted at 2026-06-19 06:52:45 on dist-test-slave-1h5s"
server_key: "230208406a26d3f111a20bebabe7de9c"
server_key_iv: "820b46aa5bff086dde26b48bf49d5fe1"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:45.613425 30285 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 06:52:45.632045 30300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:45.633935 30285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 06:52:45.634207 30285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "41012378b41442e7b1a3ec533e5ea053"
format_stamp: "Formatted at 2026-06-19 06:52:45 on dist-test-slave-1h5s"
server_key: "230208406a26d3f111a20bebabe7de9c"
server_key_iv: "820b46aa5bff086dde26b48bf49d5fe1"
server_key_version: "kuduclusterkey@0"
I20260619 06:52:45.634557 30285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:52:45.667502 30285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:52:45.671833 30285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:52:45.672169 30285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:52:45.673202 30285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:52:45.674744 30285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:52:45.674845 30285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:45.674995 30285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:52:45.675066 30285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:52:45.734050 30285 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:34687
I20260619 06:52:45.734061 30412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:34687 every 8 connection(s)
I20260619 06:52:45.735656 30285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:52:45.740991 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 30285
I20260619 06:52:45.741191 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:52:45.741710 18589 external_mini_cluster.cc:1428] Setting key 0928226a400cf9db3b8821c181cdf4b6
Jun 19 06:52:45 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851965, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:52:45.759917 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.738681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:40089 (local address 127.18.39.126:37967)
0619 06:52:45.739195 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.739202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.740353 (+ 1151us) server_negotiation.cc:407] Connection header received
0619 06:52:45.742676 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.742682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.742846 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.742961 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:45.744949 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.745752 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.748491 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.748948 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.749052 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.753177 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:45.753183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:45.753186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:45.753222 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:45.756468 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.757493 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.757500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.757503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.757676 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.758182 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.758187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.758189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.758515 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:45.758645 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.759473 (+ 828us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.759642 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":245,"threads_started":1}
I20260619 06:52:45.760926 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.739357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:37967 (local address 127.18.39.67:40089)
0619 06:52:45.740178 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:45.740224 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:45.742328 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:45.743270 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:45.743282 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:45.743936 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:52:45.744752 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.744777 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.745912 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.745916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.748297 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.748314 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.749191 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.749197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.749573 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.750391 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:52:45.750422 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:52:45.752902 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:52:45.756623 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:45.756632 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:45.756654 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:45.757347 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:45.757811 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:52:45.757815 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:52:45.757817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:52:45.758033 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:52:45.758633 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:52:45.758639 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:52:45.758990 (+ 351us) client_negotiation.cc:769] Sending connection context
0619 06:52:45.759285 (+ 295us) client_negotiation.cc:240] Negotiation successful
0619 06:52:45.759742 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":349,"threads_started":1}
I20260619 06:52:45.762478 30413 heartbeater.cc:347] Connected to a master server at 127.18.39.126:37967
I20260619 06:52:45.762768 30413 heartbeater.cc:464] Registering TS with master...
I20260619 06:52:45.763453 30413 heartbeater.cc:511] Master 127.18.39.126:37967 requested a full tablet report, sending...
I20260619 06:52:45.764863 29923 ts_manager.cc:194] Registered new tserver with Master: 41012378b41442e7b1a3ec533e5ea053 (127.18.39.67:34687)
I20260619 06:52:45.765735 29923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:40089
I20260619 06:52:45.770148 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:52:45.787603 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.774759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.18.39.126:37967)
0619 06:52:45.774937 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.774941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.775427 (+ 486us) server_negotiation.cc:407] Connection header received
0619 06:52:45.775556 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.775560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.775637 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.775727 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:45.777661 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.778397 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.779777 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.780234 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.780347 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.782862 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:45.782867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:45.782870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:45.782911 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:45.785273 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.785986 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.785990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.785993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.786103 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:45.786449 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:45.786453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:45.786455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:45.786734 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:45.786927 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.787211 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.787380 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:52:45.791966 29923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:52:45.799468 29923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:52:45.837415 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.812723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.18.39.67:34687)
0619 06:52:45.825993 (+ 13270us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.826008 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.826061 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:52:45.827046 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.827052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.827248 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.827445 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:52:45.828738 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.831251 (+ 2513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.834710 (+ 3459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.835674 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.835828 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.836071 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.836895 (+ 824us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.837076 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13031,"thread_start_us":12634,"threads_started":1}
I20260619 06:52:45.837538 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.812635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51262 (local address 127.18.39.66:42473)
0619 06:52:45.826517 (+ 13882us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.826524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.826879 (+ 355us) server_negotiation.cc:407] Connection header received
0619 06:52:45.827072 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.827079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.827376 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.827590 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:52:45.828805 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.831115 (+ 2310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.834545 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.835988 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.836103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.836267 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.836985 (+ 718us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.837159 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13727,"thread_start_us":13600,"threads_started":1}
I20260619 06:52:45.837998 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.812233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:34687 (local address 127.0.0.1:47950)
0619 06:52:45.825612 (+ 13379us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:45.825686 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:45.825961 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:45.827798 (+ 1837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:45.827805 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:45.827835 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:52:45.828440 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.828457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.831527 (+ 3070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.831535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.834552 (+ 3017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.834566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.836300 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.836310 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.836594 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.836654 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 06:52:45.837399 (+ 745us) client_negotiation.cc:240] Negotiation successful
0619 06:52:45.837554 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12958,"mutex_wait_us":641,"thread_start_us":14711,"threads_started":1}
I20260619 06:52:45.838377 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.812461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:42473 (local address 127.0.0.1:51262)
0619 06:52:45.826732 (+ 14271us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:45.826749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:45.826877 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:45.827930 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:45.827941 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:45.827981 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:52:45.828572 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.828589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.831367 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.831377 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.834354 (+ 2977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.834368 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.836421 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.836429 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.836636 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.836684 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 06:52:45.836793 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 06:52:45.837816 (+ 1023us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14158,"mutex_wait_us":803,"thread_start_us":13758,"threads_started":1}
I20260619 06:52:45.841966 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.827409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:33311 (local address 127.0.0.1:33406)
0619 06:52:45.833001 (+ 5592us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:45.833026 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:45.833162 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:45.833973 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:45.833978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:45.834002 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:52:45.834577 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.834586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.836452 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.836457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.838663 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:45.838701 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.841147 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.841155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:45.841313 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.841369 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:52:45.841462 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 06:52:45.841588 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5457,"mutex_wait_us":3,"thread_start_us":5304,"threads_started":1}
I20260619 06:52:45.842294 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:45.827715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33406 (local address 127.18.39.65:33311)
0619 06:52:45.832683 (+ 4968us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:45.832693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:45.833291 (+ 598us) server_negotiation.cc:407] Connection header received
0619 06:52:45.833426 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:45.833436 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:45.833715 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:45.833920 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:52:45.834756 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.836271 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.838863 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:45.840281 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:45.840429 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:45.841105 (+ 676us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:45.841543 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 06:52:45.841707 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4786,"thread_start_us":4629,"threads_started":1}
I20260619 06:52:45.842582 30348 tablet_service.cc:1467] Processing CreateTablet for tablet be25840863f14b2a93f2331dc58010b0 (DEFAULT_TABLE table=test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:52:45.842612 30207 tablet_service.cc:1467] Processing CreateTablet for tablet be25840863f14b2a93f2331dc58010b0 (DEFAULT_TABLE table=test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:52:45.843988 30348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25840863f14b2a93f2331dc58010b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:52:45.843988 30207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25840863f14b2a93f2331dc58010b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:52:45.846009 30071 tablet_service.cc:1467] Processing CreateTablet for tablet be25840863f14b2a93f2331dc58010b0 (DEFAULT_TABLE table=test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:52:45.847417 30071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be25840863f14b2a93f2331dc58010b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:52:45.857182 30432 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Bootstrap starting.
I20260619 06:52:45.861063 30432 tablet_bootstrap.cc:654] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Neither blocks nor log segments found. Creating new log.
I20260619 06:52:45.862059 30433 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Bootstrap starting.
I20260619 06:52:45.862990 30432 log.cc:824] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Log is configured to *not* fsync() on all Append() calls
I20260619 06:52:45.864545 30434 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Bootstrap starting.
I20260619 06:52:45.865998 30433 tablet_bootstrap.cc:654] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Neither blocks nor log segments found. Creating new log.
I20260619 06:52:45.867727 30434 tablet_bootstrap.cc:654] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Neither blocks nor log segments found. Creating new log.
I20260619 06:52:45.867748 30433 log.cc:824] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Log is configured to *not* fsync() on all Append() calls
I20260619 06:52:45.868749 30432 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: No bootstrap required, opened a new log
I20260619 06:52:45.868925 30434 log.cc:824] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Log is configured to *not* fsync() on all Append() calls
I20260619 06:52:45.869076 30432 ts_tablet_manager.cc:1397] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260619 06:52:45.870959 30433 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: No bootstrap required, opened a new log
I20260619 06:52:45.871266 30433 ts_tablet_manager.cc:1397] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 06:52:45.872123 30434 tablet_bootstrap.cc:492] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: No bootstrap required, opened a new log
I20260619 06:52:45.872891 30434 ts_tablet_manager.cc:1397] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260619 06:52:45.877828 30432 raft_consensus.cc:348] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.878224 30432 raft_consensus.cc:374] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:52:45.878345 30432 raft_consensus.cc:729] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d7d019daaf4b06be5f47842d0a2072, State: Initialized, Role: FOLLOWER
I20260619 06:52:45.878979 30432 consensus_queue.cc:260] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.880136 30279 heartbeater.cc:503] Master 127.18.39.126:37967 was elected leader, sending a full tablet report...
I20260619 06:52:45.880688 30432 ts_tablet_manager.cc:1428] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260619 06:52:45.881857 30433 raft_consensus.cc:348] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.882273 30433 raft_consensus.cc:374] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:52:45.882393 30433 raft_consensus.cc:729] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41012378b41442e7b1a3ec533e5ea053, State: Initialized, Role: FOLLOWER
I20260619 06:52:45.883003 30433 consensus_queue.cc:260] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.884073 30413 heartbeater.cc:503] Master 127.18.39.126:37967 was elected leader, sending a full tablet report...
I20260619 06:52:45.884775 30433 ts_tablet_manager.cc:1428] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260619 06:52:45.884670 30434 raft_consensus.cc:348] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.885108 30434 raft_consensus.cc:374] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:52:45.885205 30434 raft_consensus.cc:729] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127f276df5c8416496a000d9a30e7962, State: Initialized, Role: FOLLOWER
I20260619 06:52:45.885892 30434 consensus_queue.cc:260] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:45.888588 30434 ts_tablet_manager.cc:1428] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260619 06:52:45.980691 30137 tablet.cc:2369] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:52:45.987365 30414 tablet.cc:2369] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:52:46.055779 30439 raft_consensus.cc:482] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:52:46.056119 30439 raft_consensus.cc:504] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:46.057861 30439 leader_election.cc:288] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4d7d019daaf4b06be5f47842d0a2072 (127.18.39.66:42473), 127f276df5c8416496a000d9a30e7962 (127.18.39.65:33311)
I20260619 06:52:46.064847 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.058365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:51525 (local address 127.18.39.65:33311)
0619 06:52:46.058622 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.058630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.058683 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:52:46.058955 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.058962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.059057 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.059166 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:52:46.059632 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.060452 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.062844 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.064054 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.064176 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.064231 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.064567 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.064661 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:52:46.064946 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.058360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:33311 (local address 127.18.39.67:51525)
0619 06:52:46.058588 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:46.058610 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:46.058780 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:46.059202 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:46.059207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:46.059244 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:52:46.059463 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:46.059477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.061007 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.061012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:46.062599 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:46.062610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.064191 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.064197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:46.064386 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.064421 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:52:46.064541 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 06:52:46.064682 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260619 06:52:46.066088 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.058303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:33553 (local address 127.18.39.66:42473)
0619 06:52:46.058510 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.058516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.059418 (+ 902us) server_negotiation.cc:407] Connection header received
0619 06:52:46.059527 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.059534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.059634 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.059729 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:52:46.060293 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.061748 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.063928 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.065033 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.065193 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.065269 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.065818 (+ 549us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.065911 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:52:46.066283 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.058360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:42473 (local address 127.18.39.67:33553)
0619 06:52:46.059305 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0619 06:52:46.059325 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:52:46.059424 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:52:46.059753 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:52:46.059790 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:52:46.059814 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:52:46.060142 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:46.060149 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.061925 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.061928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:46.063775 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:52:46.063790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.065298 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.065303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:52:46.065615 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.065639 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:52:46.065790 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 06:52:46.065930 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":180,"thread_start_us":434,"threads_started":1}
I20260619 06:52:46.066284 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25840863f14b2a93f2331dc58010b0" candidate_uuid: "41012378b41442e7b1a3ec533e5ea053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127f276df5c8416496a000d9a30e7962" is_pre_election: true
I20260619 06:52:46.066931 30091 raft_consensus.cc:2454] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41012378b41442e7b1a3ec533e5ea053 in term 0.
I20260619 06:52:46.067303 30234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25840863f14b2a93f2331dc58010b0" candidate_uuid: "41012378b41442e7b1a3ec533e5ea053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d7d019daaf4b06be5f47842d0a2072" is_pre_election: true
I20260619 06:52:46.067718 30304 leader_election.cc:302] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127f276df5c8416496a000d9a30e7962, 41012378b41442e7b1a3ec533e5ea053; no voters:
I20260619 06:52:46.067783 30234 raft_consensus.cc:2454] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41012378b41442e7b1a3ec533e5ea053 in term 0.
I20260619 06:52:46.068084 30439 raft_consensus.cc:2781] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:52:46.068240 30439 raft_consensus.cc:482] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:52:46.068332 30439 raft_consensus.cc:3037] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:52:46.070600 30439 raft_consensus.cc:504] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:46.071225 30439 leader_election.cc:288] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [CANDIDATE]: Term 1 election: Requested vote from peers c4d7d019daaf4b06be5f47842d0a2072 (127.18.39.66:42473), 127f276df5c8416496a000d9a30e7962 (127.18.39.65:33311)
I20260619 06:52:46.071764 30234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25840863f14b2a93f2331dc58010b0" candidate_uuid: "41012378b41442e7b1a3ec533e5ea053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4d7d019daaf4b06be5f47842d0a2072"
I20260619 06:52:46.071821 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be25840863f14b2a93f2331dc58010b0" candidate_uuid: "41012378b41442e7b1a3ec533e5ea053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127f276df5c8416496a000d9a30e7962"
I20260619 06:52:46.071966 30234 raft_consensus.cc:3037] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:52:46.072036 30091 raft_consensus.cc:3037] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:52:46.074905 30234 raft_consensus.cc:2454] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41012378b41442e7b1a3ec533e5ea053 in term 1.
I20260619 06:52:46.074949 30091 raft_consensus.cc:2454] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41012378b41442e7b1a3ec533e5ea053 in term 1.
I20260619 06:52:46.075295 30303 leader_election.cc:302] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41012378b41442e7b1a3ec533e5ea053, c4d7d019daaf4b06be5f47842d0a2072; no voters:
I20260619 06:52:46.075603 30439 raft_consensus.cc:2781] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:52:46.076015 30439 raft_consensus.cc:686] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 1 LEADER]: Becoming Leader. State: Replica: 41012378b41442e7b1a3ec533e5ea053, State: Running, Role: LEADER
I20260619 06:52:46.076480 30439 consensus_queue.cc:237] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } }
I20260619 06:52:46.081285 29923 catalog_manager.cc:5495] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41012378b41442e7b1a3ec533e5ea053 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "41012378b41442e7b1a3ec533e5ea053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41012378b41442e7b1a3ec533e5ea053" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 } health_report { overall_health: UNKNOWN } } }
W20260619 06:52:46.124769 30280 tablet.cc:2369] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:52:46.143656 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.137987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.18.39.67:34687)
0619 06:52:46.138183 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.138190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.138256 (+ 66us) server_negotiation.cc:407] Connection header received
0619 06:52:46.138515 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.138519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.138605 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.138760 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:52:46.139146 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.139877 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.141363 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.141772 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.141909 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.142502 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:52:46.143078 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:52:46.143238 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.143295 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.143389 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:52:46.151986 30234 raft_consensus.cc:1261] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 1 FOLLOWER]: Refusing update from remote peer 41012378b41442e7b1a3ec533e5ea053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:52:46.151978 30091 raft_consensus.cc:1261] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 1 FOLLOWER]: Refusing update from remote peer 41012378b41442e7b1a3ec533e5ea053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:52:46.152946 30439 consensus_queue.cc:1035] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4d7d019daaf4b06be5f47842d0a2072" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:52:46.153263 30442 consensus_queue.cc:1035] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "127f276df5c8416496a000d9a30e7962" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 33311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:52:46.164161 30445 mvcc.cc:205] Tried to move back new op lower bound from 7298465653347913728 to 7298465653053231104. Current Snapshot: MvccSnapshot[applied={T|T < 7298465653347913728}]
I20260619 06:52:46.171020 30447 mvcc.cc:205] Tried to move back new op lower bound from 7298465653347913728 to 7298465653053231104. Current Snapshot: MvccSnapshot[applied={T|T < 7298465653347913728}]
I20260619 06:52:46.189697 29922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:52:46.190001 29922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:52:46.193905 29922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70353d71b8084f369cbaae51eb93f650: Sending DeleteTablet for 3 replicas of tablet be25840863f14b2a93f2331dc58010b0
I20260619 06:52:46.195456 30207 tablet_service.cc:1514] Processing DeleteTablet for tablet be25840863f14b2a93f2331dc58010b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:52:46 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:51262
I20260619 06:52:46.195456 30071 tablet_service.cc:1514] Processing DeleteTablet for tablet be25840863f14b2a93f2331dc58010b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:52:46 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:33406
I20260619 06:52:46.196308 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 30003
I20260619 06:52:46.196215 30348 tablet_service.cc:1514] Processing DeleteTablet for tablet be25840863f14b2a93f2331dc58010b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:52:46 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:47950
I20260619 06:52:46.201849 30456 tablet_replica.cc:330] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: stopping tablet replica
I20260619 06:52:46.202399 30456 raft_consensus.cc:2229] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:52:46.203008 30456 raft_consensus.cc:2258] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:52:46.205862 30456 ts_tablet_manager.cc:1905] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:52:46.213174 30456 ts_tablet_manager.cc:1918] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:52:46.213898 30457 tablet_replica.cc:330] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: stopping tablet replica
I20260619 06:52:46.214349 30457 raft_consensus.cc:2229] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:52:46.215046 30457 raft_consensus.cc:2258] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:52:46.215446 30456 log.cc:1196] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/be25840863f14b2a93f2331dc58010b0
I20260619 06:52:46.216001 30456 ts_tablet_manager.cc:1939] T be25840863f14b2a93f2331dc58010b0 P c4d7d019daaf4b06be5f47842d0a2072: Deleting consensus metadata
I20260619 06:52:46.217681 30457 ts_tablet_manager.cc:1905] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:52:46.217932 29909 catalog_manager.cc:4841] TS c4d7d019daaf4b06be5f47842d0a2072 (127.18.39.66:42473): tablet be25840863f14b2a93f2331dc58010b0 (table test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]) successfully deleted
I20260619 06:52:46.218698 30455 tablet_replica.cc:330] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: stopping tablet replica
I20260619 06:52:46.219134 30455 raft_consensus.cc:2229] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:52:46.219653 30455 raft_consensus.cc:2258] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:52:46.221987 30455 ts_tablet_manager.cc:1905] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:52:46.225656 30457 ts_tablet_manager.cc:1918] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:52:46.225906 30457 log.cc:1196] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/be25840863f14b2a93f2331dc58010b0
I20260619 06:52:46.226516 30457 ts_tablet_manager.cc:1939] T be25840863f14b2a93f2331dc58010b0 P 41012378b41442e7b1a3ec533e5ea053: Deleting consensus metadata
I20260619 06:52:46.228394 29910 catalog_manager.cc:4841] TS 41012378b41442e7b1a3ec533e5ea053 (127.18.39.67:34687): tablet be25840863f14b2a93f2331dc58010b0 (table test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]) successfully deleted
I20260619 06:52:46.229701 30455 ts_tablet_manager.cc:1918] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:52:46.229868 30455 log.cc:1196] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/be25840863f14b2a93f2331dc58010b0
I20260619 06:52:46.230459 30455 ts_tablet_manager.cc:1939] T be25840863f14b2a93f2331dc58010b0 P 127f276df5c8416496a000d9a30e7962: Deleting consensus metadata
I20260619 06:52:46.232336 29910 catalog_manager.cc:4841] TS 127f276df5c8416496a000d9a30e7962 (127.18.39.65:33311): tablet be25840863f14b2a93f2331dc58010b0 (table test-table [id=aeadf313f71e422c8bfd1e0d22e0ed08]) successfully deleted
Jun 19 06:52:46 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:52:46.242995 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.197390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.18.39.65:33311)
0619 06:52:46.197631 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.197640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.203872 (+ 6232us) server_negotiation.cc:407] Connection header received
0619 06:52:46.204071 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.204079 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.204207 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.204339 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:46.223829 (+ 19490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.224998 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.227085 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.227745 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.227852 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.236727 (+ 8875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:46.236741 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:46.236756 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:46.236802 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:46.239926 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.240685 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.240692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.240695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.240820 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.241209 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.241213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.241215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.241758 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:46.242063 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.242393 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.242616 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:52:46.243628 30131 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:52:46.334695 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 30003
I20260619 06:52:46.358719 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 30142
Jun 19 06:52:46 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:52:46.377010 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.360133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.18.39.66:42473)
0619 06:52:46.360394 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.360402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.360449 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:52:46.360628 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.360634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.360716 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.360828 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:46.362761 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.363890 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.365200 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.365867 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.365999 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.370796 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:46.370812 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:46.370829 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:46.370870 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:46.373909 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.374806 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.374813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.374815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.374938 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.375324 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.375327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.375329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.375780 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:46.376085 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.376407 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.376642 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260619 06:52:46.377816 30274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:52:46.469959 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 30142
I20260619 06:52:46.491990 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 30285
Jun 19 06:52:46 dist-test-slave-1h5s krb5kdc[29447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:52:46.508136 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:52:46.492768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.18.39.67:34687)
0619 06:52:46.493014 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:52:46.493020 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:52:46.493049 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:52:46.493160 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:52:46.493166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:52:46.493238 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:52:46.493363 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:52:46.494675 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.495492 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.496788 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:52:46.497237 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:52:46.497538 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:52:46.501998 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:52:46.502012 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:52:46.502032 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:52:46.502071 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:52:46.505168 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.505937 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.505952 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.505955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.506112 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:52:46.506494 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:52:46.506509 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:52:46.506511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:52:46.506894 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:52:46.507240 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 06:52:46.507538 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 06:52:46.507757 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 06:52:46.508955 30408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:52:46.634260 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 30285
I20260619 06:52:46.654038 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 29891
I20260619 06:52:46.655222 29953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:52:46.773277 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 29891
I20260619 06:52:46.795004 18589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 06:52:47.130901 18589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 06:52:47.131098 18589 mini_ranger.cc:66] Stopping Ranger...
I20260619 06:52:47.477002 18589 mini_ranger.cc:68] Stopped Ranger
2026-06-19 06:52:47.477 UTC [29477] LOG: received smart shutdown request
2026-06-19 06:52:47.480 UTC [29477] LOG: background worker "logical replication launcher" (PID 29486) exited with exit code 1
2026-06-19 06:52:47.480 UTC [29481] LOG: shutting down
2026-06-19 06:52:47.635 UTC [29477] LOG: database system is shut down
2026-06-19T06:52:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49670 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:52:47 dist-test-slave-1h5s krb5kdc[30471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:52:47 dist-test-slave-1h5s krb5kdc[30471](info): set up 2 sockets
Jun 19 06:52:47 dist-test-slave-1h5s krb5kdc[30471](info): commencing operation
krb5kdc: starting...
W20260619 06:52:48.873832 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:52:49 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:52:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangeradmin_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangeradmin_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangerlookup_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangerlookup_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
I20260619 06:52:49.193133 18589 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskHLTYaR/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/postgres -l logfile start
2026-06-19 06:52:51.640 UTC [30501] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 06:52:51.640 UTC [30501] LOG: listening on IPv4 address "127.18.39.84", port 42903
2026-06-19 06:52:51.644 UTC [30501] LOG: listening on Unix socket "/tmp/.s.PGSQL.42903"
2026-06-19 06:52:51.664 UTC [30504] LOG: database system was shut down at 2026-06-19 06:52:50 UTC
2026-06-19 06:52:51.668 UTC [30501] LOG: database system is ready to accept connections
I20260619 06:52:52.634002 18589 mini_postgres.cc:96] Postgres bound to 42903
2026-06-19 06:52:52.650 UTC [30512] FATAL: database "slave" does not exist
127.18.39.84:42903 - accepting connections
I20260619 06:52:52.651480 18589 mini_ranger.cc:161] Starting Ranger...
I20260619 06:52:52.676607 18589 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 06:52:53.144709 18589 mini_ranger.cc:87] Created ranger Postgres database
I20260619 06:52:53.144826 18589 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin
2026-06-19 06:52:53,442 [I] DB FLAVOR :POSTGRES
2026-06-19 06:52:53,442 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:52:53,442 [I] Checking connection..
2026-06-19 06:52:53,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 06:52:53,698 [I] Checking connection passed.
2026-06-19 06:52:53,698 [I] --------- Verifying version history table ---------
2026-06-19 06:52:53,698 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:53,956 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:52:53,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:54,225 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:52:54,225 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 06:52:54,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 06:52:54.471 UTC [30576] WARNING: there is no transaction in progress
2026-06-19 06:52:54,477 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 06:52:54,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:52:54,714 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 06:52:54,714 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:52:54,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 06:52:54,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 06:52:55,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1h5s.c.gcp-upstream.internal','N') ;"
2026-06-19 06:52:55,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:52:55,735 [I] Table x_portal_user does not exist in database ranger
2026-06-19 06:52:55,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:52:55,978 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 06:52:55,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:52:56,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:52:56,457 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 06:52:56,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 06:52:57.409 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.426 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.443 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.460 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.476 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.493 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.551 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.558 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.572 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.582 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.592 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.606 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.615 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.622 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.628 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:57.634 UTC [30684] WARNING: there is no transaction in progress
2026-06-19 06:52:58,244 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 06:52:58,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 06:52:58,497 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 06:52:58,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:52:58,754 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 06:52:58,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:52:58,992 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 06:52:58,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:52:59,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:52:59,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1h5s.c.gcp-upstream.internal';"
2026-06-19 06:52:59,758 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 06:52:59,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 06:53:00,004 [I] DB_PATCHES have already been applied
I20260619 06:53:00.010223 18589 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskHLTYaR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 06:53:00.010331 18589 mini_ranger.cc:193] Using host: 127.18.39.84
I20260619 06:53:00.025257 18589 mini_ranger.cc:227] Ranger admin URL: http://127.18.39.84:53449
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:53:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 06:53:01.151582 18589 mini_ranger.cc:160] Time spent starting Ranger: real 8.500s user 0.000s sys 0.038s
Jun 19, 2026 6:53:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:53:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 6:53:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:53:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:53:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 06:53:23.617018 18589 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.39.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.39.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangerkms_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.39.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/rangerkms_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.39.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.39.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.39.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/HTTP_127.18.39.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 06:53:23 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852003, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 06:53:23.793869 18589 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 06:53:23.815277 18589 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 06:53:24.651135 18589 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 06:53:24.651261 18589 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms
I20260619 06:53:24.651310 18589 mini_ranger_kms.cc:227] Using postgres at 127.18.39.84:42903
2026-06-19 06:53:25,010 [I] DB FLAVOR :POSTGRES
2026-06-19 06:53:25,011 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:53:25,011 [I] Checking connection
2026-06-19 06:53:25,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 06:53:25,285 [I] connection success
2026-06-19 06:53:25,285 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 06:53:25,285 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 06:53:25,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 06:53:25,552 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 06:53:25,552 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:53:25,552 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 06:53:25,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.39.84:42903/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 06:53:25,848 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 06:53:26.315128 18589 mini_ranger_kms.cc:310] Created kms service
I20260619 06:53:26.532011 18589 mini_ranger_kms.cc:326] Created kudu user
I20260619 06:53:26.581744 18589 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 06:53:26 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852003, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:26.992677 18589 mini_ranger_kms.cc:384] Added ranger policy
I20260619 06:53:26.993014 18589 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms:/tmp/dist-test-taskHLTYaR/build/asan/bin/postgresql.jar:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskHLTYaR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 06:53:26.993091 18589 mini_ranger_kms.cc:240] Using host: 127.18.39.84
I20260619 06:53:27.008774 18589 mini_ranger_kms.cc:276] Ranger KMS PID: 30865
I20260619 06:53:27.008914 18589 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.39.84:55401
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:53:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 06:53:29.331478 18589 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.538s user 0.084s sys 0.290s
I20260619 06:53:29.331724 18589 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 06:53:29.331902 18589 mini_ranger_kms.cc:401] 127.18.39.84:55401/kms/v1/keys
Jun 19 06:53:29 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781852009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 06:53:29 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:53:30 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852009, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.39.84@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:53:32 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:53:32.741004 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36233
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:36233
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:33.080571 30915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:33.080935 30915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:33.081103 30915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:33.088536 30915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:33.088704 30915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:33.088821 30915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:33.088917 30915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:33.089004 30915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:33.099736 30915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.39.84:55401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:36233
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:33.101167 30915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:33.102633 30915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:33.114763 30923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:33.114828 30920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:33.114806 30921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:33.115664 30915 server_base.cc:1034] running on GCE node
I20260619 06:53:33.116482 30915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:33.118088 30915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:33.119302 30915 hybrid_clock.cc:648] HybridClock initialized: now 1781852013119258 us; error 33 us; skew 500 ppm
I20260619 06:53:33.119693 30915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:33 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852013, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:33.126250 30915 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:33.127866 30915 webserver.cc:460] Webserver started at http://127.18.39.126:45921/ using document root <none> and password file <none>
I20260619 06:53:33.128568 30915 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:33.128670 30915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:33.128929 30915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:53:33 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852013, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:33.189952 30915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "21a668f0d1e443a1b29c61f6e943f1cd"
format_stamp: "Formatted at 2026-06-19 06:53:33 on dist-test-slave-1h5s"
server_key: "4b17cf2e3ac0332fb98a7da51a1dad32"
server_key_iv: "84a6e0c4af032dcc4fcc283550bf988e"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:33.191075 30915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "21a668f0d1e443a1b29c61f6e943f1cd"
format_stamp: "Formatted at 2026-06-19 06:53:33 on dist-test-slave-1h5s"
server_key: "4b17cf2e3ac0332fb98a7da51a1dad32"
server_key_iv: "84a6e0c4af032dcc4fcc283550bf988e"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:33.197746 30915 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260619 06:53:33.214859 30930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:33.216507 30915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:53:33.216725 30915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "21a668f0d1e443a1b29c61f6e943f1cd"
format_stamp: "Formatted at 2026-06-19 06:53:33 on dist-test-slave-1h5s"
server_key: "4b17cf2e3ac0332fb98a7da51a1dad32"
server_key_iv: "84a6e0c4af032dcc4fcc283550bf988e"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:33.216964 30915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:33.251360 30915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:33.255620 30915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:33.255962 30915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:33.275210 30915 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:36233
I20260619 06:53:33.275231 30981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:36233 every 8 connection(s)
I20260619 06:53:33.276647 30915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:33.278028 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 30915
I20260619 06:53:33.278242 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:53:33.279075 18589 external_mini_cluster.cc:1428] Setting key 613de50410ea190593a0578f30378718
I20260619 06:53:33.281554 30982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:33.292275 30982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Bootstrap starting.
Jun 19 06:53:33 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:33.295778 30982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:33.296852 30982 log.cc:824] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:33.299914 30982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: No bootstrap required, opened a new log
I20260619 06:53:33.300836 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:33.281169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.18.39.126:36233)
0619 06:53:33.281960 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:33.281972 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:33.282011 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:53:33.283677 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:33.283697 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:33.284077 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:33.284521 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:33.285989 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:33.287300 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:33.288623 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:33.289316 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:33.289631 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:33.293538 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:33.293546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:33.293561 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:33.293613 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:33.296607 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:33.297398 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:33.297405 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:33.297412 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:33.297654 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:33.298137 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:33.298141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:33.298146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:33.298659 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:33.298959 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:33.299404 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 06:53:33.299799 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":394,"threads_started":1}
I20260619 06:53:33.308152 30982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:33.308743 30982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:33.308874 30982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a668f0d1e443a1b29c61f6e943f1cd, State: Initialized, Role: FOLLOWER
I20260619 06:53:33.309567 30982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:33.309814 30982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:33.309964 30982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:33.310151 30982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:33.312407 30982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:33.312996 30982 leader_election.cc:302] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a668f0d1e443a1b29c61f6e943f1cd; no voters:
I20260619 06:53:33.313750 30982 leader_election.cc:288] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:53:33.313900 30987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:33.314731 30987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 LEADER]: Becoming Leader. State: Replica: 21a668f0d1e443a1b29c61f6e943f1cd, State: Running, Role: LEADER
I20260619 06:53:33.315267 30987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:33.316039 30982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:33.318320 30989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a668f0d1e443a1b29c61f6e943f1cd. Latest consensus state: current_term: 1 leader_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } } }
I20260619 06:53:33.318485 30988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } } }
I20260619 06:53:33.318806 30988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:33.318812 30989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:33.320562 30996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:33.326632 30996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:33.327455 30982 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 06:53:33.346441 30996 catalog_manager.cc:1347] Generated new cluster ID: a654f0d4bbf8474ebdc1d8de9b46dd2d
I20260619 06:53:33.346588 30996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:33.377956 30996 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:53:33.380016 30996 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:33.388950 30996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Generated new TSK 0
I20260619 06:53:33.390129 30996 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:53:34.120705 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:34 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 06:53:34.530468 31027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:34.530817 31027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:34.530985 31027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:34.539021 31027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:34.539196 31027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:34.539390 31027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:53:34.550367 31027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:34.551774 31027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:34.553164 31027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:34.571122 31038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:34.571532 31037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:34.577744 31027 server_base.cc:1034] running on GCE node
W20260619 06:53:34.578521 31040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:34.579396 31027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:34.580744 31027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:34.582047 31027 hybrid_clock.cc:648] HybridClock initialized: now 1781852014581979 us; error 55 us; skew 500 ppm
I20260619 06:53:34.582461 31027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:34 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:34.586962 31027 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:53:34.589393 31027 webserver.cc:460] Webserver started at http://127.18.39.65:33473/ using document root <none> and password file <none>
I20260619 06:53:34.590345 31027 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:34.590487 31027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:34.590869 31027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:53:34 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:34.615434 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "0410637d163d48f98c185f24d3359b5d"
format_stamp: "Formatted at 2026-06-19 06:53:34 on dist-test-slave-1h5s"
server_key: "2a9594c230b4763130b4575d07ee9523"
server_key_iv: "ffacb27429fa4c330547bf81b2020b2a"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:34.616613 31027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "0410637d163d48f98c185f24d3359b5d"
format_stamp: "Formatted at 2026-06-19 06:53:34 on dist-test-slave-1h5s"
server_key: "2a9594c230b4763130b4575d07ee9523"
server_key_iv: "ffacb27429fa4c330547bf81b2020b2a"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:34.624436 31027 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260619 06:53:34.654628 31047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:34.656766 31027 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.004s sys 0.008s
I20260619 06:53:34.657074 31027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "0410637d163d48f98c185f24d3359b5d"
format_stamp: "Formatted at 2026-06-19 06:53:34 on dist-test-slave-1h5s"
server_key: "2a9594c230b4763130b4575d07ee9523"
server_key_iv: "ffacb27429fa4c330547bf81b2020b2a"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:34.657473 31027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:34.710029 31027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:34.714555 31027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:34.714908 31027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:34.715938 31027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:34.717543 31027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:34.717635 31027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:34.717782 31027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:34.717841 31027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:34.780375 31027 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:36811
I20260619 06:53:34.780408 31159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:36811 every 8 connection(s)
I20260619 06:53:34.782198 31027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:53:34.790529 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31027
I20260619 06:53:34.790750 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:53:34.791608 18589 external_mini_cluster.cc:1428] Setting key 00bfbee81a9e5c1b1a9e7d772dc4bf09
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Jun 19 06:53:34 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:34.849112 31162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:34.786060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.65:55161)
0619 06:53:34.812876 (+ 26816us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:34.812959 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:34.816668 (+ 3709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:34.827249 (+ 10581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:34.827265 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:34.827958 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:34.828880 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:34.828912 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:34.831175 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:34.831181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:34.833321 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:34.833364 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:34.835899 (+ 2535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:34.835908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:34.836242 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:34.837108 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:34.837148 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:34.840082 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:34.843970 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:34.843989 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:34.844012 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:34.844640 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:34.845115 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:34.845123 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:34.845130 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:34.845413 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:34.846161 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:34.846172 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:34.846678 (+ 506us) client_negotiation.cc:769] Sending connection context
0619 06:53:34.846980 (+ 302us) client_negotiation.cc:240] Negotiation successful
0619 06:53:34.847587 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26320,"thread_start_us":29844,"threads_started":1}
I20260619 06:53:34.850636 31163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:34.787275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:55161 (local address 127.18.39.126:36233)
0619 06:53:34.817407 (+ 30132us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:34.817417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:34.817444 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:53:34.817531 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:34.817539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:34.817684 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:34.817823 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:34.829171 (+ 11348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:34.830939 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:34.833539 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:34.835363 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:34.835497 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:34.840362 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:34.840368 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:34.840371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:34.840413 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:34.843806 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:34.844814 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:34.844823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:34.844830 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:34.844992 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:34.845563 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:34.845571 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:34.845577 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:34.846019 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:34.849641 (+ 3622us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:34.849838 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 06:53:34.850134 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24716,"thread_start_us":29803,"threads_started":1}
I20260619 06:53:34.852386 31160 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:34.852758 31160 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:34.853631 31160 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:34.856108 30947 ts_manager.cc:194] Registered new tserver with Master: 0410637d163d48f98c185f24d3359b5d (127.18.39.65:36811)
I20260619 06:53:34.858419 30947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:55161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:53:34.873142 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:34 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
Jun 19 06:53:35 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852014, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
W20260619 06:53:35.336532 31166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:35.336963 31166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:35.337188 31166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:35.347596 31166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:35.347833 31166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:35.348047 31166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:53:35.363610 31166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:35.365406 31166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:35.367040 31166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:35.383389 31174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:35.396168 31176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:35.398648 31173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:35.414834 31166 server_base.cc:1034] running on GCE node
I20260619 06:53:35.415771 31166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:35.422005 31166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:35.423566 31166 hybrid_clock.cc:648] HybridClock initialized: now 1781852015423443 us; error 119 us; skew 500 ppm
I20260619 06:53:35.424216 31166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:35 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852015, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:35.428736 31166 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:53:35.449371 31166 webserver.cc:460] Webserver started at http://127.18.39.66:33027/ using document root <none> and password file <none>
I20260619 06:53:35.450590 31166 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:35.450836 31166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:35.451251 31166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:53:35 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852015, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:35.485728 31166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "c8f3b6ebc05447429c99435b29240e67"
format_stamp: "Formatted at 2026-06-19 06:53:35 on dist-test-slave-1h5s"
server_key: "187b2dcbca7ed9c44c2d7f5726e622f2"
server_key_iv: "edbf950532e3c3d4c63d1c24904edd21"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:35.486910 31166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "c8f3b6ebc05447429c99435b29240e67"
format_stamp: "Formatted at 2026-06-19 06:53:35 on dist-test-slave-1h5s"
server_key: "187b2dcbca7ed9c44c2d7f5726e622f2"
server_key_iv: "edbf950532e3c3d4c63d1c24904edd21"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:35.498946 31166 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.011s sys 0.001s
I20260619 06:53:35.530215 31183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:35.532791 31166 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.010s sys 0.005s
I20260619 06:53:35.537324 31166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "c8f3b6ebc05447429c99435b29240e67"
format_stamp: "Formatted at 2026-06-19 06:53:35 on dist-test-slave-1h5s"
server_key: "187b2dcbca7ed9c44c2d7f5726e622f2"
server_key_iv: "edbf950532e3c3d4c63d1c24904edd21"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:35.537753 31166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:35.614961 31166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:35.620698 31166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:35.621136 31166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:35.628693 31166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:35.637214 31166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:35.637503 31166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:53:35.637712 31166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:35.637836 31166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:35.710410 31166 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:45901
I20260619 06:53:35.710453 31302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:45901 every 8 connection(s)
I20260619 06:53:35.712206 31166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:53:35.719671 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31166
I20260619 06:53:35.719843 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:53:35.720373 18589 external_mini_cluster.cc:1428] Setting key 325107e1e054f3ee6607557d0ccc08d8
Jun 19 06:53:35 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852015, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:53:35.754580 31305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:35.715239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47141 (local address 127.18.39.126:36233)
0619 06:53:35.733554 (+ 18315us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:35.733560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:35.733586 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:53:35.733650 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:35.733656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:35.733774 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:35.733905 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:35.735789 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:35.737057 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:35.739910 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:35.740714 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:35.740843 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:35.745985 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:35.745995 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:35.746004 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:35.746057 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:35.750265 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:35.751436 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:35.751445 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:35.751452 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:35.751640 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:35.752259 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:35.752264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:35.752268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:35.752682 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:35.752830 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:35.753909 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0619 06:53:35.754152 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18172,"thread_start_us":18028,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:53:35.756024 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:35.715778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.66:47141)
0619 06:53:35.720032 (+ 4254us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:35.720102 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:35.733007 (+ 12905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:35.734254 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:35.734270 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:35.734957 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:35.735573 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:35.735591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:35.737296 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:35.737305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:35.739693 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:35.739716 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:35.741236 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:35.741245 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:35.741666 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:35.742600 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:35.742635 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:35.745672 (+ 3037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:35.750535 (+ 4863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:35.750547 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:35.750571 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:35.751274 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:35.751815 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:35.751822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:35.751829 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:35.752107 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:35.752927 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:35.752935 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:35.753296 (+ 361us) client_negotiation.cc:769] Sending connection context
0619 06:53:35.753804 (+ 508us) client_negotiation.cc:240] Negotiation successful
0619 06:53:35.754432 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3820,"thread_start_us":16154,"threads_started":1}
I20260619 06:53:35.758201 31303 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:35.758530 31303 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:35.759408 31303 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:35.761162 30947 ts_manager.cc:194] Registered new tserver with Master: c8f3b6ebc05447429c99435b29240e67 (127.18.39.66:45901)
I20260619 06:53:35.762440 30947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:47141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:53:35.784406 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:35.862782 31160 heartbeater.cc:503] Master 127.18.39.126:36233 was elected leader, sending a full tablet report...
W20260619 06:53:36.125074 31309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:36.125463 31309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:36.125625 31309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:36.133225 31309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:36.133414 31309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:36.133585 31309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:53:36.143977 31309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:36.145488 31309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:36.146791 31309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:36.160418 31314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:36.160368 31315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:36.160339 31317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:36.162882 31309 server_base.cc:1034] running on GCE node
I20260619 06:53:36.163502 31309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:36.164791 31309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:36.166024 31309 hybrid_clock.cc:648] HybridClock initialized: now 1781852016165981 us; error 41 us; skew 500 ppm
I20260619 06:53:36.166359 31309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852016, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:36.169646 31309 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:53:36.171352 31309 webserver.cc:460] Webserver started at http://127.18.39.67:34387/ using document root <none> and password file <none>
I20260619 06:53:36.172091 31309 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:36.172238 31309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:36.172545 31309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852016, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:36.189767 31309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "76316cf577984dca942768b42f61b139"
format_stamp: "Formatted at 2026-06-19 06:53:36 on dist-test-slave-1h5s"
server_key: "f3154fe42171a1f05761b424c7335bb4"
server_key_iv: "8f6747d71053835cb440eec60cc83b2b"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:36.190603 31309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "76316cf577984dca942768b42f61b139"
format_stamp: "Formatted at 2026-06-19 06:53:36 on dist-test-slave-1h5s"
server_key: "f3154fe42171a1f05761b424c7335bb4"
server_key_iv: "8f6747d71053835cb440eec60cc83b2b"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:36.196223 31309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:53:36.213815 31324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:36.215447 31309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:53:36.215631 31309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "76316cf577984dca942768b42f61b139"
format_stamp: "Formatted at 2026-06-19 06:53:36 on dist-test-slave-1h5s"
server_key: "f3154fe42171a1f05761b424c7335bb4"
server_key_iv: "8f6747d71053835cb440eec60cc83b2b"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:36.215852 31309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:36.243534 31309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:36.248003 31309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:36.248320 31309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:36.249282 31309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:36.250802 31309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:36.250907 31309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:36.251058 31309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:36.251130 31309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:36.285305 31309 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:41435
I20260619 06:53:36.285347 31436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:41435 every 8 connection(s)
I20260619 06:53:36.286619 31309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:53:36.293226 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31309
I20260619 06:53:36.293434 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:53:36.293974 18589 external_mini_cluster.cc:1428] Setting key d93f65ce0b5b8bda7d4b9e0eed19719e
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852016, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:36.307061 31439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:36.289119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34539 (local address 127.18.39.126:36233)
0619 06:53:36.289653 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:36.289659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:36.290601 (+ 942us) server_negotiation.cc:407] Connection header received
0619 06:53:36.292558 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:36.292564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:36.292668 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:36.292790 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:36.294468 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.295459 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.297228 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.297710 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.297841 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:36.301581 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:36.301587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:36.301591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:36.301624 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:36.304053 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.304754 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.304762 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.304770 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.304890 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.305325 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.305351 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.305357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.305635 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:36.305792 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:36.306461 (+ 669us) server_negotiation.cc:299] Negotiation successful
0619 06:53:36.306688 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":283,"threads_started":1}
I20260619 06:53:36.308097 31440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:36.289780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.67:34539)
0619 06:53:36.290447 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:36.290483 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:36.292265 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:36.293048 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:36.293057 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:36.293635 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:36.294288 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:36.294311 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.295600 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.295609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:36.297084 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:36.297096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.297927 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.297935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:36.298181 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:36.298832 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:36.298854 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:36.301295 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:36.304198 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:36.304207 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:36.304224 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:36.304631 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:36.304995 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:36.305001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:36.305005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:36.305207 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:36.305763 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:36.305770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:36.306103 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 06:53:36.306372 (+ 269us) client_negotiation.cc:240] Negotiation successful
0619 06:53:36.306893 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":267,"threads_started":1}
I20260619 06:53:36.309687 31437 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:36.310009 31437 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:36.310707 31437 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:36.312152 30947 ts_manager.cc:194] Registered new tserver with Master: 76316cf577984dca942768b42f61b139 (127.18.39.67:41435)
I20260619 06:53:36.312963 30947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:34539
I20260619 06:53:36.322320 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:53:36.324038 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31027
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:53:36.340363 31442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:36.324965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.18.39.65:36811)
0619 06:53:36.325604 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:36.325612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:36.325637 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:53:36.325694 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:36.325701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:36.325912 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:36.326045 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:36.327592 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.328589 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.330072 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.330636 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.330737 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:36.334862 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:36.334874 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:36.334889 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:36.334923 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:36.337868 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.338533 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.338540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.338542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.338685 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.339011 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.339014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.339016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.339351 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:36.339575 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:36.339813 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 06:53:36.340070 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":326,"threads_started":1}
I20260619 06:53:36.341732 31155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:36.428117 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31027
I20260619 06:53:36.443846 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31166
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:53:36.460479 31444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:36.444703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.18.39.66:45901)
0619 06:53:36.445237 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:36.445244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:36.445271 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:53:36.445379 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:36.445394 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:36.445615 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:36.445755 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:36.446917 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.447929 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.449543 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.450096 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.450199 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:36.454765 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:36.454780 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:36.454797 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:36.454834 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:36.457922 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.458610 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.458617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.458619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.458749 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.459114 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.459117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.459119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.459478 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:36.459699 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:36.459983 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 06:53:36.460182 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":300,"threads_started":1}
I20260619 06:53:36.461800 31298 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:36.545680 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31166
I20260619 06:53:36.561515 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31309
Jun 19 06:53:36 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:53:36.578363 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:36.562668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.18.39.67:41435)
0619 06:53:36.563346 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:36.563354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:36.563382 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:53:36.563447 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:36.563453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:36.563712 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:36.563851 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:36.565080 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.566271 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.567513 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:36.568106 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:36.568205 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:36.572773 (+ 4568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:36.572786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:36.572801 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:36.572841 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:36.575824 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.576462 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.576469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.576471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.576596 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:36.576904 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:36.576907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:36.576909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:36.577243 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:36.577534 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:36.577759 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 06:53:36.577951 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":360,"threads_started":1}
I20260619 06:53:36.579617 31432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:36.663877 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31309
I20260619 06:53:36.679487 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 30915
I20260619 06:53:36.680781 30977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:36.787853 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 30915
I20260619 06:53:36.808908 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36233
--webserver_interface=127.18.39.126
--webserver_port=45921
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:36233
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:37.140815 31449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:37.141137 31449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:37.141230 31449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:37.148912 31449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:37.149048 31449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:37.149137 31449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:37.149245 31449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:37.149318 31449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:37.160126 31449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:36233
--ranger_config_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=45921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:37.161535 31449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:37.162901 31449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:37.175434 31454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:37.175513 31455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:37.175545 31457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:37.176616 31449 server_base.cc:1034] running on GCE node
I20260619 06:53:37.177573 31449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:37.179235 31449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:37.180491 31449 hybrid_clock.cc:648] HybridClock initialized: now 1781852017180420 us; error 63 us; skew 500 ppm
I20260619 06:53:37.180869 31449 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:37 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852017, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:37.184790 31449 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:37.186645 31449 webserver.cc:460] Webserver started at http://127.18.39.126:45921/ using document root <none> and password file <none>
I20260619 06:53:37.187376 31449 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:37.187528 31449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:53:37 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852017, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:37.238647 31449 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 06:53:37.241885 31465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:37.243353 31449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:53:37.243558 31449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "21a668f0d1e443a1b29c61f6e943f1cd"
format_stamp: "Formatted at 2026-06-19 06:53:33 on dist-test-slave-1h5s"
server_key: "4b17cf2e3ac0332fb98a7da51a1dad32"
server_key_iv: "84a6e0c4af032dcc4fcc283550bf988e"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:37.244570 31449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:37.265563 31449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:37.269650 31449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:37.269953 31449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:37.288915 31449 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:36233
I20260619 06:53:37.288933 31516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:36233 every 8 connection(s)
I20260619 06:53:37.290541 31449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:37.293936 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31449
I20260619 06:53:37.294803 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:36811
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:36233
--webserver_port=33473
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:37.297729 31517 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:53:37.301065 31517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Bootstrap starting.
I20260619 06:53:37.320771 31517 log.cc:824] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:37.330089 31517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:53:37.330662 31517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Bootstrap complete.
I20260619 06:53:37.339803 31517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:37.340663 31517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a668f0d1e443a1b29c61f6e943f1cd, State: Initialized, Role: FOLLOWER
I20260619 06:53:37.341321 31517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:37.341629 31517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:37.341755 31517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:37.341946 31517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:53:37.346211 31517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:37.346789 31517 leader_election.cc:302] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21a668f0d1e443a1b29c61f6e943f1cd; no voters:
I20260619 06:53:37.347433 31517 leader_election.cc:288] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:53:37.347594 31521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:53:37.348663 31521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [term 2 LEADER]: Becoming Leader. State: Replica: 21a668f0d1e443a1b29c61f6e943f1cd, State: Running, Role: LEADER
I20260619 06:53:37.349454 31521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } }
I20260619 06:53:37.349784 31517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:37.352658 31522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } } }
I20260619 06:53:37.352952 31522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:37.352085 31523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21a668f0d1e443a1b29c61f6e943f1cd. Latest consensus state: current_term: 2 leader_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21a668f0d1e443a1b29c61f6e943f1cd" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36233 } } }
I20260619 06:53:37.353551 31523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:37.366674 31527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:37.374099 31527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:37.374941 31517 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 06:53:37.375712 31527 catalog_manager.cc:1259] Loaded cluster ID: a654f0d4bbf8474ebdc1d8de9b46dd2d
I20260619 06:53:37.375852 31527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:37.379894 31527 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:37.381626 31527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Loaded TSK: 0
I20260619 06:53:37.388940 31527 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:53:37.754998 31519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:37.755549 31519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:37.755774 31519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:37.768147 31519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:37.768473 31519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:37.768728 31519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:53:37.785400 31519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:36811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=33473
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:37.787547 31519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:37.789489 31519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:37.810951 31554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:37.813778 31555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:37.824164 31557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:37.825374 31519 server_base.cc:1034] running on GCE node
I20260619 06:53:37.826337 31519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:37.827924 31519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:37.829250 31519 hybrid_clock.cc:648] HybridClock initialized: now 1781852017829158 us; error 83 us; skew 500 ppm
I20260619 06:53:37.829721 31519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:37 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852017, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:37.834256 31519 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:53:37.836328 31519 webserver.cc:460] Webserver started at http://127.18.39.65:33473/ using document root <none> and password file <none>
I20260619 06:53:37.837131 31519 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:37.837311 31519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:53:37 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852017, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:37.875295 31519 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.005s sys 0.004s
I20260619 06:53:37.884878 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:37.887202 31519 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.001s
I20260619 06:53:37.887579 31519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "0410637d163d48f98c185f24d3359b5d"
format_stamp: "Formatted at 2026-06-19 06:53:34 on dist-test-slave-1h5s"
server_key: "2a9594c230b4763130b4575d07ee9523"
server_key_iv: "ffacb27429fa4c330547bf81b2020b2a"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:37.889366 31519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:37.959651 31519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:37.965168 31519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:37.965734 31519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:37.972765 31519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:37.975183 31519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:37.975474 31519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:53:37.975663 31519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:37.975804 31519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:38.031992 31519 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:36811
I20260619 06:53:38.032806 31678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:36811 every 8 connection(s)
I20260619 06:53:38.033788 31519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:53:38.037055 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31519
I20260619 06:53:38.037990 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:45901
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:36233
--webserver_port=33027
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:38 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852017, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:38.085381 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:38.037258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:50957 (local address 127.18.39.126:36233)
0619 06:53:38.053915 (+ 16657us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:38.053931 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:38.053982 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:53:38.056030 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:38.056050 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:38.056575 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:38.056977 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:38.059672 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.061875 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.064783 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.065988 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.066217 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:38.071812 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:38.071826 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:38.071849 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:38.071913 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:38.077599 (+ 5686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:38.078758 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:38.078772 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:38.078783 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:38.079021 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:38.079639 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:38.079644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:38.079648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:38.080312 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:38.080501 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:38.083603 (+ 3102us) server_negotiation.cc:299] Negotiation successful
0619 06:53:38.084043 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16396,"thread_start_us":16130,"threads_started":1}
I20260619 06:53:38.085906 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:38.037316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.65:50957)
0619 06:53:38.051628 (+ 14312us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:38.051697 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:38.054310 (+ 2613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:38.057784 (+ 3474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:38.057799 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:38.058576 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:38.059418 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:38.059447 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.062135 (+ 2688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.062143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:38.064564 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:38.064585 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.066272 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.066283 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:38.066611 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:38.067479 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:38.067519 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:38.071411 (+ 3892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:38.077839 (+ 6428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:38.077850 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:38.077869 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:38.078508 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:38.079193 (+ 685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:38.079201 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:38.079206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:38.079484 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:38.080600 (+ 1116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:38.080608 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:38.081033 (+ 425us) client_negotiation.cc:769] Sending connection context
0619 06:53:38.081370 (+ 337us) client_negotiation.cc:240] Negotiation successful
0619 06:53:38.082023 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13867,"thread_start_us":17268,"threads_started":1}
I20260619 06:53:38.090073 31679 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:38.090442 31679 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:38.091365 31679 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
W20260619 06:53:38.094921 31679 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:36233 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:53:38.095170 31679 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:38.095469 31679 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:38.096828 31679 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:38.097187 31679 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
W20260619 06:53:38.098405 31679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:53:38.464216 31683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:38.464555 31683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:38.464745 31683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:38.472924 31683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:38.473106 31683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:38.473289 31683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:53:38.484561 31683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:45901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=33027
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:38.486129 31683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:38.487494 31683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:38.503618 31695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:38.504166 31683 server_base.cc:1034] running on GCE node
W20260619 06:53:38.506204 31696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:38.506342 31698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:38.507226 31683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:38.508482 31683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:38.509693 31683 hybrid_clock.cc:648] HybridClock initialized: now 1781852018509616 us; error 66 us; skew 500 ppm
I20260619 06:53:38.510134 31683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:38 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:38.517437 31683 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:53:38.520231 31683 webserver.cc:460] Webserver started at http://127.18.39.66:33027/ using document root <none> and password file <none>
I20260619 06:53:38.521087 31683 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:38.521281 31683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:53:38 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:38.544018 31683 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:53:38.557530 31708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:38.559437 31683 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.005s
I20260619 06:53:38.559657 31683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "c8f3b6ebc05447429c99435b29240e67"
format_stamp: "Formatted at 2026-06-19 06:53:35 on dist-test-slave-1h5s"
server_key: "187b2dcbca7ed9c44c2d7f5726e622f2"
server_key_iv: "edbf950532e3c3d4c63d1c24904edd21"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:38.560778 31683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:38.594026 31683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:38.600111 31683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:38.600533 31683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:38.609469 31683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:38.611979 31683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:38.612169 31683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:53:38.612339 31683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:38.612449 31683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:38.681649 31683 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:45901
I20260619 06:53:38.681774 31822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:45901 every 8 connection(s)
I20260619 06:53:38.683614 31683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:53:38.693825 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31683
I20260619 06:53:38.694638 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.39.84:55401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:41435
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:36233
--webserver_port=34387
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:38 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 06:53:38 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:38.750159 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:38.688588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:55375 (local address 127.18.39.126:36233)
0619 06:53:38.705987 (+ 17399us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:38.705995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:38.706025 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:53:38.706105 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:38.706113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:38.706872 (+ 759us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:38.707016 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:38.710407 (+ 3391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.720795 (+ 10388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.728653 (+ 7858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.729408 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.729556 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:38.740815 (+ 11259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:38.740823 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:38.740827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:38.740888 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:38.744368 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:38.745412 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:38.745418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:38.745421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:38.745576 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:38.746176 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:38.746181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:38.746184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:38.746557 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:38.746703 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:38.749568 (+ 2865us) server_negotiation.cc:299] Negotiation successful
0619 06:53:38.749825 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17227,"thread_start_us":17625,"threads_started":1}
I20260619 06:53:38.750496 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:38.687868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.66:55375)
0619 06:53:38.692275 (+ 4407us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:38.692323 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:38.695242 (+ 2919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:38.707752 (+ 12510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:38.707766 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:38.708424 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:38.709706 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:38.709739 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.721171 (+ 11432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.721182 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:38.723940 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:38.723966 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:38.729935 (+ 5969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:38.729943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:38.730263 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:38.737587 (+ 7324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:38.737630 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:38.740454 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:38.744588 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:38.744597 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:38.744614 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:38.745214 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:38.745757 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:38.745762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:38.745765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:38.746023 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:38.746795 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:38.746803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:38.747163 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 06:53:38.747532 (+ 369us) client_negotiation.cc:240] Negotiation successful
0619 06:53:38.748100 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4105,"thread_start_us":3891,"threads_started":1}
I20260619 06:53:38.752197 31823 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:38.752518 31823 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:38.753309 31823 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:38.755505 31482 ts_manager.cc:194] Registered new tserver with Master: c8f3b6ebc05447429c99435b29240e67 (127.18.39.66:45901)
I20260619 06:53:38.757695 31482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:55375
I20260619 06:53:39.098541 31679 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:39.098969 31679 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:39.100225 31482 ts_manager.cc:194] Registered new tserver with Master: 0410637d163d48f98c185f24d3359b5d (127.18.39.65:36811)
I20260619 06:53:39.101503 31482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:50957
W20260619 06:53:39.128237 31827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:39.128643 31827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:39.128856 31827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:39.140239 31827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:39.140390 31827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:39.140617 31827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:53:39.153720 31827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.39.84:55401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:41435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=34387
--tserver_master_addrs=127.18.39.126:36233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:39.155089 31827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:39.156507 31827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Jun 19 06:53:39 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
W20260619 06:53:39.190928 31834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:39.191375 31833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:39.194700 31836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:39.195928 31827 server_base.cc:1034] running on GCE node
I20260619 06:53:39.196751 31827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:39.208388 31827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:39.216148 31827 hybrid_clock.cc:648] HybridClock initialized: now 1781852019216012 us; error 100 us; skew 500 ppm
I20260619 06:53:39.216576 31827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:39 dist-test-slave-1h5s krb5kdc[30471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852019, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:39.221325 31827 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:53:39.261801 31827 webserver.cc:460] Webserver started at http://127.18.39.67:34387/ using document root <none> and password file <none>
I20260619 06:53:39.262980 31827 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:39.263230 31827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:53:39 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852019, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:39.326018 31827 fs_manager.cc:709] Time spent opening directory manager: real 0.032s user 0.011s sys 0.017s
I20260619 06:53:39.339362 31844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:39.341564 31827 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.008s sys 0.006s
I20260619 06:53:39.341887 31827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "76316cf577984dca942768b42f61b139"
format_stamp: "Formatted at 2026-06-19 06:53:36 on dist-test-slave-1h5s"
server_key: "f3154fe42171a1f05761b424c7335bb4"
server_key_iv: "8f6747d71053835cb440eec60cc83b2b"
server_key_version: "kuduclusterkey@0"
I20260619 06:53:39.343434 31827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:39.412954 31827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:39.418200 31827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:39.418541 31827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:39.425959 31827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:39.434201 31827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:39.434444 31827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:53:39.434620 31827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:39.434746 31827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:53:39 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852018, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for HTTP/127.18.39.84@KRBTEST.COM
I20260619 06:53:39.762318 31823 heartbeater.cc:503] Master 127.18.39.126:36233 was elected leader, sending a full tablet report...
I20260619 06:53:39.837831 31827 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:41435
I20260619 06:53:39.837848 31957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:41435 every 8 connection(s)
I20260619 06:53:39.839573 31827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:53:39.848326 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 31827
Jun 19 06:53:39 dist-test-slave-1h5s krb5kdc[30471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852019, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:39.874219 31961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.843564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:36233 (local address 127.18.39.67:48679)
0619 06:53:39.844429 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:39.844480 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:39.846210 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:39.846936 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:39.846945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:39.847421 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:39.847994 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.848010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.851163 (+ 3153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.851170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.853375 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.853395 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.854868 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.854878 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.855202 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.856053 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:39.856092 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:39.858709 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:39.863252 (+ 4543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:39.863263 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:39.863284 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:39.863932 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:39.864406 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:39.864411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:39.864414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:39.864683 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:39.871321 (+ 6638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:39.871332 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:39.871877 (+ 545us) client_negotiation.cc:769] Sending connection context
0619 06:53:39.872223 (+ 346us) client_negotiation.cc:240] Negotiation successful
0619 06:53:39.872785 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":373,"threads_started":1}
I20260619 06:53:39.875419 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.843239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:48679 (local address 127.18.39.126:36233)
0619 06:53:39.843807 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.843816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.844621 (+ 805us) server_negotiation.cc:407] Connection header received
0619 06:53:39.846495 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.846502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.846603 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.846744 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:39.848194 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.850930 (+ 2736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.853586 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.854300 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.854411 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.859023 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:39.859031 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:39.859037 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:39.859096 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:39.863082 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.864094 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.864100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.864104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.864266 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.864822 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.864828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.864831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.865209 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:39.865402 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.874772 (+ 9370us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.875035 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
I20260619 06:53:39.880257 31958 heartbeater.cc:347] Connected to a master server at 127.18.39.126:36233
I20260619 06:53:39.880627 31958 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:39.881603 31958 heartbeater.cc:511] Master 127.18.39.126:36233 requested a full tablet report, sending...
I20260619 06:53:39.883474 31482 ts_manager.cc:194] Registered new tserver with Master: 76316cf577984dca942768b42f61b139 (127.18.39.67:41435)
I20260619 06:53:39.884661 31482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:48679
I20260619 06:53:39.887094 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.849445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.18.39.126:36233)
0619 06:53:39.860643 (+ 11198us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.860650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.861324 (+ 674us) server_negotiation.cc:407] Connection header received
0619 06:53:39.861599 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.861605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.861716 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.861830 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:39.863819 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.864745 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.869457 (+ 4712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.870119 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.870259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.879280 (+ 9021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:39.879288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:39.879293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:39.879337 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:39.882965 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.883952 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.883959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.883963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.884123 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.884570 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.884576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.884579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.884928 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:39.885217 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.886475 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.886718 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11038,"thread_start_us":11216,"threads_started":1}
I20260619 06:53:39.890097 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:53:39.910617 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.894309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.18.39.126:36233)
0619 06:53:39.894511 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.894517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.895960 (+ 1443us) server_negotiation.cc:407] Connection header received
0619 06:53:39.896190 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.896195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.896281 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.896397 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:39.897997 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.898951 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.900917 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.901429 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.901531 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.904934 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:39.904940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:39.904943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:39.904982 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:39.907857 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.908691 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.908695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.908697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.908841 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:39.909265 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:39.909268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:39.909270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:39.909614 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:39.909819 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.910189 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.910367 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:53:39.916045 31482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:53:39.923580 31482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:53:39.974436 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.938586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42570 (local address 127.18.39.65:36811)
0619 06:53:39.965544 (+ 26958us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.965554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.966917 (+ 1363us) server_negotiation.cc:407] Connection header received
0619 06:53:39.967313 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.967321 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.967534 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.967718 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:39.968586 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.970028 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.971868 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.973240 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.973582 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.973831 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.973908 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.974087 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26777,"thread_start_us":26604,"threads_started":1}
I20260619 06:53:39.974449 31983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.937830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:45901 (local address 127.0.0.1:38514)
0619 06:53:39.963060 (+ 25230us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:39.963133 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:39.963361 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:39.965730 (+ 2369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:39.965741 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:39.965848 (+ 107us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:39.966438 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.966455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.968828 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.968835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.971093 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.971107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.973308 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.973314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.973570 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.973629 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 06:53:39.973774 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 06:53:39.974001 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24772,"mutex_wait_us":44,"thread_start_us":28600,"threads_started":1}
I20260619 06:53:39.974486 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.938001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.18.39.66:45901)
0619 06:53:39.964870 (+ 26869us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.964877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.964906 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:53:39.965051 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.965060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.965376 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.965620 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:39.966687 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.968695 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.971274 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.972787 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.972916 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.973163 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.973958 (+ 795us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.974115 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26660,"thread_start_us":26892,"threads_started":1}
I20260619 06:53:39.974997 31981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.938290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:36811 (local address 127.0.0.1:42570)
0619 06:53:39.966780 (+ 28490us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:39.966801 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:39.966920 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:39.967967 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:39.967972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:39.967995 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:39.968396 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.968407 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.970169 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.970174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.971718 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.971731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.973505 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.973514 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.973749 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.973782 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:53:39.973846 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 06:53:39.974738 (+ 892us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28385,"mutex_wait_us":848,"thread_start_us":25351,"threads_started":1}
I20260619 06:53:39.974992 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.938703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.18.39.67:41435)
0619 06:53:39.964721 (+ 26018us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:39.964731 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:39.966409 (+ 1678us) server_negotiation.cc:407] Connection header received
0619 06:53:39.966642 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:39.966649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:39.966886 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:39.967049 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:39.967902 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.969664 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.972163 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.973581 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.973705 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.973888 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:39.974471 (+ 583us) server_negotiation.cc:299] Negotiation successful
0619 06:53:39.974657 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25809,"thread_start_us":26026,"threads_started":1}
I20260619 06:53:39.975901 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:39.937942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41435 (local address 127.0.0.1:55330)
0619 06:53:39.966273 (+ 28331us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:39.966299 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:39.966436 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:39.967143 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:39.967147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:39.967175 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:39.967604 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.967616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.969811 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.969816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.971998 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:39.972012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:39.974033 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:39.974039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:39.974222 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:39.974259 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:53:39.975454 (+ 1195us) client_negotiation.cc:240] Negotiation successful
0619 06:53:39.975607 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27971,"thread_start_us":28163,"threads_started":1}
I20260619 06:53:39.978886 31614 tablet_service.cc:1467] Processing CreateTablet for tablet 2312a98aa1634390bcf89c17e964026b (DEFAULT_TABLE table=test-table [id=158686308cb9498ea95acd29fdfbb73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:39.979000 31758 tablet_service.cc:1467] Processing CreateTablet for tablet 2312a98aa1634390bcf89c17e964026b (DEFAULT_TABLE table=test-table [id=158686308cb9498ea95acd29fdfbb73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:39.979234 31893 tablet_service.cc:1467] Processing CreateTablet for tablet 2312a98aa1634390bcf89c17e964026b (DEFAULT_TABLE table=test-table [id=158686308cb9498ea95acd29fdfbb73c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:39.980238 31614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2312a98aa1634390bcf89c17e964026b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:39.980383 31758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2312a98aa1634390bcf89c17e964026b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:39.980429 31893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2312a98aa1634390bcf89c17e964026b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:39.992254 31986 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Bootstrap starting.
I20260619 06:53:39.996232 31988 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Bootstrap starting.
I20260619 06:53:39.996426 31987 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Bootstrap starting.
I20260619 06:53:39.996835 31986 tablet_bootstrap.cc:654] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:39.998322 31986 log.cc:824] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:39.999715 31987 tablet_bootstrap.cc:654] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:39.999831 31988 tablet_bootstrap.cc:654] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:40.000833 31988 log.cc:824] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:40.001026 31987 log.cc:824] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:40.001612 31986 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: No bootstrap required, opened a new log
I20260619 06:53:40.001936 31986 ts_tablet_manager.cc:1397] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260619 06:53:40.003930 31988 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: No bootstrap required, opened a new log
I20260619 06:53:40.004205 31988 ts_tablet_manager.cc:1397] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260619 06:53:40.005025 31987 tablet_bootstrap.cc:492] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: No bootstrap required, opened a new log
I20260619 06:53:40.005292 31987 ts_tablet_manager.cc:1397] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260619 06:53:40.009694 31986 raft_consensus.cc:348] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.010085 31986 raft_consensus.cc:374] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:40.010205 31986 raft_consensus.cc:729] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0410637d163d48f98c185f24d3359b5d, State: Initialized, Role: FOLLOWER
I20260619 06:53:40.010821 31986 consensus_queue.cc:260] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.011953 31679 heartbeater.cc:503] Master 127.18.39.126:36233 was elected leader, sending a full tablet report...
I20260619 06:53:40.012650 31986 ts_tablet_manager.cc:1428] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260619 06:53:40.015091 31988 raft_consensus.cc:348] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.015444 31988 raft_consensus.cc:374] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:40.015569 31988 raft_consensus.cc:729] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76316cf577984dca942768b42f61b139, State: Initialized, Role: FOLLOWER
I20260619 06:53:40.016185 31988 consensus_queue.cc:260] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.016049 31987 raft_consensus.cc:348] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.016480 31987 raft_consensus.cc:374] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:40.016584 31987 raft_consensus.cc:729] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f3b6ebc05447429c99435b29240e67, State: Initialized, Role: FOLLOWER
I20260619 06:53:40.017179 31987 consensus_queue.cc:260] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.017678 31958 heartbeater.cc:503] Master 127.18.39.126:36233 was elected leader, sending a full tablet report...
I20260619 06:53:40.018056 31988 ts_tablet_manager.cc:1428] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Time spent starting tablet: real 0.014s user 0.009s sys 0.006s
I20260619 06:53:40.021000 31987 ts_tablet_manager.cc:1428] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
W20260619 06:53:40.036674 31680 tablet.cc:2369] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:53:40.093551 31959 tablet.cc:2369] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:53:40.193022 31824 tablet.cc:2369] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:40.348356 31994 raft_consensus.cc:482] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:40.348748 31994 raft_consensus.cc:504] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.351138 31994 leader_election.cc:288] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76316cf577984dca942768b42f61b139 (127.18.39.67:41435), 0410637d163d48f98c185f24d3359b5d (127.18.39.65:36811)
I20260619 06:53:40.360889 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.351854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:51239 (local address 127.18.39.65:36811)
0619 06:53:40.352116 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.352124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.352481 (+ 357us) server_negotiation.cc:407] Connection header received
0619 06:53:40.352802 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.352808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.352889 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.353000 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:40.353952 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.355265 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.358295 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.359568 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.359739 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.359832 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.360482 (+ 650us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.360617 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:53:40.360984 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.351520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:36811 (local address 127.18.39.66:51239)
0619 06:53:40.352333 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:40.352352 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:40.352616 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:40.353104 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:40.353111 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:40.353147 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:40.353772 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:40.353784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.355448 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.355456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:40.358072 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:40.358091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.360051 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.360057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:40.360268 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.360309 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 06:53:40.360434 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 06:53:40.360596 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":106,"thread_start_us":444,"threads_started":1}
I20260619 06:53:40.361248 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.351513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:49607 (local address 127.18.39.67:41435)
0619 06:53:40.351742 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.351750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.352744 (+ 994us) server_negotiation.cc:407] Connection header received
0619 06:53:40.353008 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.353016 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.353103 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.353203 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:40.353972 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.355472 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.358495 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.359958 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.360138 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.360226 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.360807 (+ 581us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.360957 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:53:40.362020 31996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.351551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:41435 (local address 127.18.39.66:49607)
0619 06:53:40.352622 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:40.352643 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:40.352846 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:40.353236 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:40.353241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:40.353377 (+ 136us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:40.353773 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:40.353784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.355670 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.355676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:40.358313 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:40.358329 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.360406 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.360413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:40.360590 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.360636 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 06:53:40.361459 (+ 823us) client_negotiation.cc:240] Negotiation successful
0619 06:53:40.361608 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"mutex_wait_us":47,"spinlock_wait_cycles":40320,"thread_start_us":748,"threads_started":1}
I20260619 06:53:40.362687 31634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2312a98aa1634390bcf89c17e964026b" candidate_uuid: "c8f3b6ebc05447429c99435b29240e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0410637d163d48f98c185f24d3359b5d" is_pre_election: true
I20260619 06:53:40.363329 31634 raft_consensus.cc:2454] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8f3b6ebc05447429c99435b29240e67 in term 0.
I20260619 06:53:40.363489 31913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2312a98aa1634390bcf89c17e964026b" candidate_uuid: "c8f3b6ebc05447429c99435b29240e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76316cf577984dca942768b42f61b139" is_pre_election: true
I20260619 06:53:40.363984 31913 raft_consensus.cc:2454] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8f3b6ebc05447429c99435b29240e67 in term 0.
I20260619 06:53:40.364106 31711 leader_election.cc:302] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0410637d163d48f98c185f24d3359b5d, c8f3b6ebc05447429c99435b29240e67; no voters:
I20260619 06:53:40.364594 31994 raft_consensus.cc:2781] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:53:40.364750 31994 raft_consensus.cc:482] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:53:40.364858 31994 raft_consensus.cc:3037] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:40.367292 31994 raft_consensus.cc:504] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.367949 31994 leader_election.cc:288] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [CANDIDATE]: Term 1 election: Requested vote from peers 76316cf577984dca942768b42f61b139 (127.18.39.67:41435), 0410637d163d48f98c185f24d3359b5d (127.18.39.65:36811)
I20260619 06:53:40.368359 31913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2312a98aa1634390bcf89c17e964026b" candidate_uuid: "c8f3b6ebc05447429c99435b29240e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76316cf577984dca942768b42f61b139"
I20260619 06:53:40.368534 31634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2312a98aa1634390bcf89c17e964026b" candidate_uuid: "c8f3b6ebc05447429c99435b29240e67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0410637d163d48f98c185f24d3359b5d"
I20260619 06:53:40.368567 31913 raft_consensus.cc:3037] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:40.368701 31634 raft_consensus.cc:3037] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:40.370812 31913 raft_consensus.cc:2454] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8f3b6ebc05447429c99435b29240e67 in term 1.
I20260619 06:53:40.370903 31634 raft_consensus.cc:2454] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8f3b6ebc05447429c99435b29240e67 in term 1.
I20260619 06:53:40.371343 31711 leader_election.cc:302] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0410637d163d48f98c185f24d3359b5d, c8f3b6ebc05447429c99435b29240e67; no voters:
I20260619 06:53:40.371639 31994 raft_consensus.cc:2781] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:40.372156 31994 raft_consensus.cc:686] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 1 LEADER]: Becoming Leader. State: Replica: c8f3b6ebc05447429c99435b29240e67, State: Running, Role: LEADER
I20260619 06:53:40.372686 31994 consensus_queue.cc:237] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } }
I20260619 06:53:40.377502 31482 catalog_manager.cc:5495] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8f3b6ebc05447429c99435b29240e67 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "c8f3b6ebc05447429c99435b29240e67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8f3b6ebc05447429c99435b29240e67" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 } health_report { overall_health: UNKNOWN } } }
I20260619 06:53:40.430603 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.424677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.18.39.66:45901)
0619 06:53:40.424914 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.424922 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.425151 (+ 229us) server_negotiation.cc:407] Connection header received
0619 06:53:40.425439 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.425443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.425531 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.425713 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:53:40.426078 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.426842 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.428325 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.428730 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.428866 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.429451 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:53:40.429992 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:53:40.430152 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.430234 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.430319 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:53:40.438886 31913 raft_consensus.cc:1261] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 1 FOLLOWER]: Refusing update from remote peer c8f3b6ebc05447429c99435b29240e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:40.438902 31634 raft_consensus.cc:1261] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 1 FOLLOWER]: Refusing update from remote peer c8f3b6ebc05447429c99435b29240e67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:40.439913 31994 consensus_queue.cc:1035] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0410637d163d48f98c185f24d3359b5d" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:40.440217 31997 consensus_queue.cc:1035] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76316cf577984dca942768b42f61b139" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 41435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:40.452301 32001 mvcc.cc:205] Tried to move back new op lower bound from 7298465875707412480 to 7298465875450961920. Current Snapshot: MvccSnapshot[applied={T|T < 7298465875707412480}]
I20260619 06:53:40.455622 32003 mvcc.cc:205] Tried to move back new op lower bound from 7298465875707412480 to 7298465875450961920. Current Snapshot: MvccSnapshot[applied={T|T < 7298465875707412480}]
I20260619 06:53:40.470028 31481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:40.470295 31481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:40.474056 31481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 21a668f0d1e443a1b29c61f6e943f1cd: Sending DeleteTablet for 3 replicas of tablet 2312a98aa1634390bcf89c17e964026b
I20260619 06:53:40.475323 31893 tablet_service.cc:1514] Processing DeleteTablet for tablet 2312a98aa1634390bcf89c17e964026b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:55330
I20260619 06:53:40.475301 31614 tablet_service.cc:1514] Processing DeleteTablet for tablet 2312a98aa1634390bcf89c17e964026b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:42570
I20260619 06:53:40.475461 31758 tablet_service.cc:1514] Processing DeleteTablet for tablet 2312a98aa1634390bcf89c17e964026b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38514
I20260619 06:53:40.476209 32010 tablet_replica.cc:330] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: stopping tablet replica
I20260619 06:53:40.476709 32010 raft_consensus.cc:2229] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:40.477262 32010 raft_consensus.cc:2258] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:40.479775 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31519
I20260619 06:53:40.481047 32010 ts_tablet_manager.cc:1905] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:40.488554 32011 tablet_replica.cc:330] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: stopping tablet replica
I20260619 06:53:40.489199 32011 raft_consensus.cc:2229] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:40.489969 32012 tablet_replica.cc:330] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: stopping tablet replica
I20260619 06:53:40.490155 32010 ts_tablet_manager.cc:1918] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:40.490399 32010 log.cc:1196] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/2312a98aa1634390bcf89c17e964026b
I20260619 06:53:40.490504 32012 raft_consensus.cc:2229] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:53:40.491120 32010 ts_tablet_manager.cc:1939] T 2312a98aa1634390bcf89c17e964026b P 76316cf577984dca942768b42f61b139: Deleting consensus metadata
I20260619 06:53:40.491412 32012 raft_consensus.cc:2258] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:40.492357 32011 raft_consensus.cc:2258] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:40.493285 31467 catalog_manager.cc:4841] TS 76316cf577984dca942768b42f61b139 (127.18.39.67:41435): tablet 2312a98aa1634390bcf89c17e964026b (table test-table [id=158686308cb9498ea95acd29fdfbb73c]) successfully deleted
I20260619 06:53:40.494840 32011 ts_tablet_manager.cc:1905] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:40.494877 32012 ts_tablet_manager.cc:1905] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:40.502586 32012 ts_tablet_manager.cc:1918] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:40.502588 32011 ts_tablet_manager.cc:1918] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:40.502784 32012 log.cc:1196] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/2312a98aa1634390bcf89c17e964026b
I20260619 06:53:40.502813 32011 log.cc:1196] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/2312a98aa1634390bcf89c17e964026b
I20260619 06:53:40.503368 32012 ts_tablet_manager.cc:1939] T 2312a98aa1634390bcf89c17e964026b P c8f3b6ebc05447429c99435b29240e67: Deleting consensus metadata
I20260619 06:53:40.503368 32011 ts_tablet_manager.cc:1939] T 2312a98aa1634390bcf89c17e964026b P 0410637d163d48f98c185f24d3359b5d: Deleting consensus metadata
I20260619 06:53:40.505192 31468 catalog_manager.cc:4841] TS 0410637d163d48f98c185f24d3359b5d (127.18.39.65:36811): tablet 2312a98aa1634390bcf89c17e964026b (table test-table [id=158686308cb9498ea95acd29fdfbb73c]) successfully deleted
I20260619 06:53:40.505213 31469 catalog_manager.cc:4841] TS c8f3b6ebc05447429c99435b29240e67 (127.18.39.66:45901): tablet 2312a98aa1634390bcf89c17e964026b (table test-table [id=158686308cb9498ea95acd29fdfbb73c]) successfully deleted
I20260619 06:53:40.513608 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.481104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42586 (local address 127.18.39.65:36811)
0619 06:53:40.481833 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.481841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.492865 (+ 11024us) server_negotiation.cc:407] Connection header received
0619 06:53:40.493079 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.493086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.493201 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.493323 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:40.499398 (+ 6075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.500559 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.502690 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.503353 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.503458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.507522 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:40.507534 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:40.507549 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:40.507585 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:40.510521 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.511402 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.511410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.511412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.511541 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.511943 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.511947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.511949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.512293 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:40.512521 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.512878 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.513139 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591}
I20260619 06:53:40.514292 31674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:40.607244 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31519
I20260619 06:53:40.629961 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31683
I20260619 06:53:40.646361 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.630737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.18.39.66:45901)
0619 06:53:40.630917 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.630924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.631011 (+ 87us) server_negotiation.cc:407] Connection header received
0619 06:53:40.631198 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.631202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.631278 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.631363 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:40.633112 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.634062 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.635965 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.636406 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.636498 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.639904 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:40.639923 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:40.639941 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:40.639989 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:40.643326 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.644197 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.644204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.644206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.644362 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.644755 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.644758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.644760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.645117 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:40.645608 (+ 491us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.645868 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.646041 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:53:40.647145 31818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:40.760624 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31683
I20260619 06:53:40.778654 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31827
I20260619 06:53:40.792253 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:40.779468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.18.39.67:41435)
0619 06:53:40.779648 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:40.779653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:40.779670 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:53:40.779815 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:40.779819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:40.779883 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:40.779963 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:40.781441 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.782139 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.783495 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:40.783888 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:40.783984 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:40.786679 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:40.786692 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:40.786708 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:40.786760 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:40.789776 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.790428 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.790436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.790438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.790567 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:40.790898 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:40.790901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:40.790903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:40.791255 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:40.791463 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:40.791723 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:53:40.791883 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:53:40.792873 31953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:40.883385 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31827
I20260619 06:53:40.899425 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 31449
I20260619 06:53:40.900658 31512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:41.015110 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 31449
I20260619 06:53:41.036882 18589 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 06:53:41.372774 18589 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 06:53:41.373024 18589 mini_ranger.cc:66] Stopping Ranger...
I20260619 06:53:41.719130 18589 mini_ranger.cc:68] Stopped Ranger
2026-06-19 06:53:41.719 UTC [30501] LOG: received smart shutdown request
2026-06-19 06:53:41.722 UTC [30501] LOG: background worker "logical replication launcher" (PID 30510) exited with exit code 1
2026-06-19 06:53:41.722 UTC [30505] LOG: shutting down
2026-06-19 06:53:41.871 UTC [30501] LOG: database system is shut down
2026-06-19T06:53:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54249 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:53:42 dist-test-slave-1h5s krb5kdc[32026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:53:42 dist-test-slave-1h5s krb5kdc[32026](info): set up 2 sockets
Jun 19 06:53:42 dist-test-slave-1h5s krb5kdc[32026](info): commencing operation
krb5kdc: starting...
W20260619 06:53:43.121838 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:53:43 dist-test-slave-1h5s krb5kdc[32026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:53:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:53:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:53:43.359063 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:33283
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:43.705806 32039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:43.706152 32039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:43.706337 32039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:43.713889 32039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:43.714013 32039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:43.714108 32039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:43.714217 32039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:43.714301 32039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:43.724586 32039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:33283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:43.726038 32039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:43.727385 32039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:43.739900 32044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:43.739868 32045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:43.740252 32047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:43.741016 32039 server_base.cc:1034] running on GCE node
I20260619 06:53:43.741966 32039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:43.743554 32039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:43.744801 32039 hybrid_clock.cc:648] HybridClock initialized: now 1781852023744718 us; error 70 us; skew 500 ppm
I20260619 06:53:43.745286 32039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:43 dist-test-slave-1h5s krb5kdc[32026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:43.753460 32039 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:43.755138 32039 webserver.cc:460] Webserver started at http://127.18.39.126:36927/ using document root <none> and password file <none>
I20260619 06:53:43.755908 32039 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:43.756062 32039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:43.756387 32039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:43.759577 32039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad"
format_stamp: "Formatted at 2026-06-19 06:53:43 on dist-test-slave-1h5s"
server_key: "59db02412e12cc2ab215b58ae870ad5c"
server_key_iv: "9c7251ea3a50b6a270b2771a2a67b7a6"
server_key_version: "encryptionkey@0"
I20260619 06:53:43.760504 32039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad"
format_stamp: "Formatted at 2026-06-19 06:53:43 on dist-test-slave-1h5s"
server_key: "59db02412e12cc2ab215b58ae870ad5c"
server_key_iv: "9c7251ea3a50b6a270b2771a2a67b7a6"
server_key_version: "encryptionkey@0"
I20260619 06:53:43.766155 32039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:53:43.769830 32054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:43.771327 32039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:53:43.771528 32039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad"
format_stamp: "Formatted at 2026-06-19 06:53:43 on dist-test-slave-1h5s"
server_key: "59db02412e12cc2ab215b58ae870ad5c"
server_key_iv: "9c7251ea3a50b6a270b2771a2a67b7a6"
server_key_version: "encryptionkey@0"
I20260619 06:53:43.771772 32039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:43.792053 32039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:43.796594 32039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:43.796907 32039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:43.816956 32039 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:33283
I20260619 06:53:43.816980 32105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:33283 every 8 connection(s)
I20260619 06:53:43.818454 32039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:43.822324 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32039
I20260619 06:53:43.822466 32106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:43.822576 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:53:43.823361 18589 external_mini_cluster.cc:1428] Setting key 73f1286b0438e600983f9fa0c25a8776
I20260619 06:53:43.829566 32106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.39.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.125@KRBTEST.COM" created.
Jun 19 06:53:43 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:43.858439 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.833752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.18.39.126:33283)
0619 06:53:43.835853 (+ 2101us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:43.835859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:43.835905 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:53:43.838292 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:43.838298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:43.838564 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:43.838746 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:43.840490 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:43.841855 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:43.844130 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:43.844804 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:43.844949 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:43.848902 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:43.848917 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:43.848940 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:43.848990 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:43.853291 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:43.854259 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:43.854272 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:43.854279 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:43.854495 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:43.854994 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:43.855001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:43.855005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:43.855585 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:43.855808 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:43.856371 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 06:53:43.856845 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":1907,"spinlock_wait_cycles":57216,"thread_start_us":1673,"threads_started":1}
I20260619 06:53:43.858457 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.833752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:33283 (local address 127.0.0.1:43910)
0619 06:53:43.834842 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.834898 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:43.837830 (+ 2932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:43.838831 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:43.838842 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:43.839712 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:43.840220 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:43.840249 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:43.842027 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:43.842035 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:43.843963 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:43.843981 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:43.845039 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:43.845045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:43.845180 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:43.845928 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:43.845951 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:43.848619 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:43.853504 (+ 4885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:43.853519 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:43.853535 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:43.854100 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:43.854624 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:43.854631 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:43.854637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:43.854859 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:43.855746 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:43.855755 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:43.855978 (+ 223us) client_negotiation.cc:769] Sending connection context
0619 06:53:43.856248 (+ 270us) client_negotiation.cc:240] Negotiation successful
0619 06:53:43.856964 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":213,"thread_start_us":501,"threads_started":1}
I20260619 06:53:43.862030 32106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:53:43.863309 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.862878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:47986)
0619 06:53:43.863064 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.863181 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 06:53:43.863646 32056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
I20260619 06:53:43.865489 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.864960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:47994)
0619 06:53:43.865172 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.865276 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:53:43.865770 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:46625: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
I20260619 06:53:43.877033 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:46625
--webserver_interface=127.18.39.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:43.915164 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } attempt: 1
I20260619 06:53:43.916445 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.915955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48004)
0619 06:53:43.916156 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.916314 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 06:53:43.917577 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.917174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48012)
0619 06:53:43.917358 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.917445 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 06:53:43.917775 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:46625: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
I20260619 06:53:43.986035 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } attempt: 2
I20260619 06:53:43.987442 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.986815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48014)
0619 06:53:43.987129 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.987240 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 06:53:43.988577 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:43.988158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48028)
0619 06:53:43.988317 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:43.988418 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:53:43.988811 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:46625: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
I20260619 06:53:44.080081 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } attempt: 3
I20260619 06:53:44.081387 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.080765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48036)
0619 06:53:44.081007 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.081141 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 06:53:44.082499 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.082089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48038)
0619 06:53:44.082247 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.082333 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:53:44.082727 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:46625: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
W20260619 06:53:44.213940 32113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:44.214253 32113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:44.214342 32113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:44.221727 32113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:44.221845 32113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:44.221890 32113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:44.221939 32113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:44.221983 32113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:44.225984 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } attempt: 4
I20260619 06:53:44.227263 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.226643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48052)
0619 06:53:44.226969 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.227080 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260619 06:53:44.228333 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.227908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48064)
0619 06:53:44.228058 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.228158 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:53:44.228524 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:46625: Network error: Client connection negotiation failed: client connection to 127.18.39.125:46625: connect: Connection refused (error 111)
I20260619 06:53:44.233035 32113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:46625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.39.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:44.234431 32113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:44.235740 32113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:44.247845 32118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:44.247813 32119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:44.247920 32121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:44.248828 32113 server_base.cc:1034] running on GCE node
I20260619 06:53:44.249749 32113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:44.251282 32113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:44.252527 32113 hybrid_clock.cc:648] HybridClock initialized: now 1781852024252463 us; error 50 us; skew 500 ppm
I20260619 06:53:44.252928 32113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:44.256358 32113 init.cc:377] Logged in from keytab as kudu/127.18.39.125@KRBTEST.COM (short username kudu)
I20260619 06:53:44.257968 32113 webserver.cc:460] Webserver started at http://127.18.39.125:45673/ using document root <none> and password file <none>
I20260619 06:53:44.258652 32113 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:44.258833 32113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:44.259107 32113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:44.262284 32113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data/instance:
uuid: "e80c298f60e64883a810e54506e7b02d"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "3ef1c3b366e2dac53c7ae031944b0ac2"
server_key_iv: "99cc139fe97d7a2fe8529a0f2b6ad5c7"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.263085 32113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal/instance:
uuid: "e80c298f60e64883a810e54506e7b02d"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "3ef1c3b366e2dac53c7ae031944b0ac2"
server_key_iv: "99cc139fe97d7a2fe8529a0f2b6ad5c7"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.268455 32113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:53:44.272221 32128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:44.275100 32113 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:53:44.275319 32113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "e80c298f60e64883a810e54506e7b02d"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "3ef1c3b366e2dac53c7ae031944b0ac2"
server_key_iv: "99cc139fe97d7a2fe8529a0f2b6ad5c7"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.275591 32113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:44.296999 32113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:44.301714 32113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:44.301964 32113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:44.319967 32179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.125:46625 every 8 connection(s)
I20260619 06:53:44.319968 32113 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.125:46625
I20260619 06:53:44.321483 32113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
I20260619 06:53:44.325424 32180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:44.330389 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32113
I20260619 06:53:44.330586 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-1/wal/instance
I20260619 06:53:44.331069 18589 external_mini_cluster.cc:1428] Setting key 14dbe9994cc8f0ef1650ca1bbe6120e8
I20260619 06:53:44.332509 32180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.124@KRBTEST.COM" created.
I20260619 06:53:44.359828 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.335966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.18.39.126:33283)
0619 06:53:44.336174 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.336181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.337825 (+ 1644us) server_negotiation.cc:407] Connection header received
0619 06:53:44.340098 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.340104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.340203 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.340322 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.341998 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.342665 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.344250 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.344675 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.344842 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.348626 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.348631 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.348634 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.348669 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.352523 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.355472 (+ 2949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.355500 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.355511 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.355795 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.356564 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.356573 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.356579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.357531 (+ 952us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.357699 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.358888 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.359206 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:53:44.360862 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.336702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:33283 (local address 127.0.0.1:43912)
0619 06:53:44.337615 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.337668 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.339784 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.340579 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.340588 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.341150 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.341819 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.341838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.342849 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.342853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.344088 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.344100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.344952 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.344959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.345196 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.346000 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.346024 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.348370 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.354334 (+ 5964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.354346 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.354368 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.355190 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.355998 (+ 808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.356007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.356012 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.356359 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.357792 (+ 1433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.357802 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.358387 (+ 585us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.358751 (+ 364us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.359382 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":371,"threads_started":1}
I20260619 06:53:44.363056 32180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:53:44.391109 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.364194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48066)
0619 06:53:44.369747 (+ 5553us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.369784 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.369945 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.374951 (+ 5006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.374956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.375545 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.375769 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.375798 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.378152 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.378158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.379595 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.379617 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.381020 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.381026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.381375 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.382165 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.382183 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.384304 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.388383 (+ 4079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.388387 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.388390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.388767 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.389194 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.389197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.389200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.389420 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.390220 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.390224 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.390376 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.390479 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.390761 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5349,"mutex_wait_us":1}
I20260619 06:53:44.391297 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.364711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48066 (local address 127.18.39.125:46625)
0619 06:53:44.374228 (+ 9517us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.374238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.374293 (+ 55us) server_negotiation.cc:407] Connection header received
0619 06:53:44.374383 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.374393 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.374731 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.374949 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.375994 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.377957 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.379788 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.380860 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.380997 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.384570 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.384589 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.384609 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.384668 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.388197 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.388949 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.388956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.388958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.389085 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.389575 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.389580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.389583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.390084 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.390295 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.390651 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.390860 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9294,"thread_start_us":9130,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
I20260619 06:53:44.393913 32180 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:53:44.395052 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:46755
--webserver_interface=127.18.39.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:44.395256 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.394710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49760)
0619 06:53:44.394936 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.395052 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260619 06:53:44.395573 32129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.397215 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.396865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49772)
0619 06:53:44.397018 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.397103 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:53:44.397480 32180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.446849 32180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 1
I20260619 06:53:44.448133 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.447580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49786)
0619 06:53:44.447825 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.447950 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:53:44.449236 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.448851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49794)
0619 06:53:44.449017 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.449105 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 06:53:44.449496 32180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.517796 32180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 2
I20260619 06:53:44.519191 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.518561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49798)
0619 06:53:44.518846 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.518979 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 06:53:44.520328 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.519931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49804)
0619 06:53:44.520099 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.520188 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:53:44.520603 32180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.527825 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } attempt: 5
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:53:44.543262 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.528665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48074)
0619 06:53:44.528872 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.528900 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.529005 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.529434 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.529437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.529822 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.530017 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.530026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.531415 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.531419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.532377 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.532389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.533447 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.533452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.533610 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.534320 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.534338 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.536740 (+ 2402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.540780 (+ 4040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.540784 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.540787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.541203 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.541660 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.541663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.541665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.541810 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.542490 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.542493 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.542627 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.542759 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.543007 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 06:53:44.543591 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.528817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48074 (local address 127.18.39.125:46625)
0619 06:53:44.529017 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.529023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.529045 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:53:44.529111 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.529116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.529206 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.529322 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.530202 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.531260 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.532522 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.533202 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.533377 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.536978 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.536984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.536989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.537032 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.540593 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.541378 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.541384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.541388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.541538 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.541920 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.541925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.541928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.542368 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.542526 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.542895 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.543175 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:53:44.544579 32106 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:53:44.545904 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.545480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49818)
0619 06:53:44.545648 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.545772 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 06:53:44.546957 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.546568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49832)
0619 06:53:44.546778 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.546859 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260619 06:53:44.547165 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.598389 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 1
I20260619 06:53:44.599553 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.599069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49838)
0619 06:53:44.599277 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.599382 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:53:44.600548 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.600225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49852)
0619 06:53:44.600383 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.600448 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:53:44.600775 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.611855 32180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 3
I20260619 06:53:44.613199 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.612515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49868)
0619 06:53:44.612823 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.612963 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 06:53:44.614562 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.613975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49870)
0619 06:53:44.614262 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.614382 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260619 06:53:44.614871 32180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.668993 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 2
I20260619 06:53:44.670308 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.669702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49878)
0619 06:53:44.669993 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.670111 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:53:44.671429 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.671010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49894)
0619 06:53:44.671197 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.671289 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 06:53:44.671643 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
W20260619 06:53:44.738380 32187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:44.738672 32187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:44.738765 32187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:44.746233 32187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:44.746351 32187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:44.746397 32187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:44.746454 32187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:44.746497 32187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:44.756839 32187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:33283,127.18.39.125:46625,127.18.39.124:46755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:46755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.39.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:44.758184 32187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:44.758117 32180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 4
I20260619 06:53:44.759387 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.758825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49900)
0619 06:53:44.759079 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.759185 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:53:44.759824 32187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 06:53:44.760504 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.760128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49904)
0619 06:53:44.760288 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.760377 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:53:44.760758 32180 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
W20260619 06:53:44.772164 32195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:44.772248 32192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:44.772197 32193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:44.777858 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 3
I20260619 06:53:44.778131 32187 server_base.cc:1034] running on GCE node
I20260619 06:53:44.778918 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.778480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49918)
0619 06:53:44.778661 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.778741 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 06:53:44.779284 32187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:44.779911 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.779554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49920)
0619 06:53:44.779713 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.779785 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 06:53:44.780182 32106 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:46755: Network error: Client connection negotiation failed: client connection to 127.18.39.124:46755: connect: Connection refused (error 111)
I20260619 06:53:44.781118 32187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:44.782343 32187 hybrid_clock.cc:648] HybridClock initialized: now 1781852024782262 us; error 52 us; skew 500 ppm
I20260619 06:53:44.782745 32187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:44.785969 32187 init.cc:377] Logged in from keytab as kudu/127.18.39.124@KRBTEST.COM (short username kudu)
I20260619 06:53:44.787462 32187 webserver.cc:460] Webserver started at http://127.18.39.124:34803/ using document root <none> and password file <none>
I20260619 06:53:44.788276 32187 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:44.788447 32187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:44.788749 32187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:44.791764 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data/instance:
uuid: "e85eec1a89f3452194b9d26463379cc2"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "ecbab69b7ad6c793987f4abb7632e743"
server_key_iv: "1c90e4fe1ca0ba540a350b93d6c8c84d"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.792578 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal/instance:
uuid: "e85eec1a89f3452194b9d26463379cc2"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "ecbab69b7ad6c793987f4abb7632e743"
server_key_iv: "1c90e4fe1ca0ba540a350b93d6c8c84d"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.797971 32187 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:53:44.801492 32202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:44.802897 32187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:53:44.803088 32187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "e85eec1a89f3452194b9d26463379cc2"
format_stamp: "Formatted at 2026-06-19 06:53:44 on dist-test-slave-1h5s"
server_key: "ecbab69b7ad6c793987f4abb7632e743"
server_key_iv: "1c90e4fe1ca0ba540a350b93d6c8c84d"
server_key_version: "encryptionkey@0"
I20260619 06:53:44.803292 32187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:44.815610 32187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:44.821058 32187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:44.821455 32187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:44.840862 32187 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.124:46755
I20260619 06:53:44.840883 32253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.124:46755 every 8 connection(s)
I20260619 06:53:44.842361 32187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
I20260619 06:53:44.846266 32254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:44.848218 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32187
I20260619 06:53:44.848450 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851818886250-18589-0/minicluster-data/master-2/wal/instance
I20260619 06:53:44.848959 18589 external_mini_cluster.cc:1428] Setting key c6909cb150fcedb9b25560915c18cd69
I20260619 06:53:44.854060 32254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:44.871975 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.857146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.18.39.126:33283)
0619 06:53:44.857312 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.857316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.858219 (+ 903us) server_negotiation.cc:407] Connection header received
0619 06:53:44.858421 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.858426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.858504 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.858610 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.859683 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.860375 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.861935 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.862383 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.862517 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.865954 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.865959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.865962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.865997 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.869027 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.869933 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.869937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.869939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.870076 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.870438 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.870441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.870443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.870753 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.870920 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.871487 (+ 567us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.871721 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:53:44.873160 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.857238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:33283 (local address 127.0.0.1:43926)
0619 06:53:44.858075 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.858110 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.858290 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.858667 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.858676 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.859268 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.859538 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.859547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.860538 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.860542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.861773 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.861790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.862608 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.862614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.862766 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.863383 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.863405 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.865737 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.869167 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.869176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.869189 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.869811 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.870180 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.870185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.870188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.870334 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.870874 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.870879 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.871134 (+ 255us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.871323 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.871782 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":134,"spinlock_wait_cycles":22400,"thread_start_us":382,"threads_started":1}
I20260619 06:53:44.873180 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.850997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.18.39.124:46755)
0619 06:53:44.851995 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.852008 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.852054 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:53:44.853843 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.853868 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.854308 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.854660 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.856583 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.858655 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.860073 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.860748 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.860954 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.865312 (+ 4358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.865322 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.865377 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.865422 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.868316 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.869173 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.869180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.869185 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.869425 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.869789 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.869794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.869797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.870493 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.870731 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.871183 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.871642 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":766,"thread_start_us":500,"threads_started":1}
I20260619 06:53:44.874917 32254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:44.890578 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.875881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:46625 (local address 127.0.0.1:48076)
0619 06:53:44.876033 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.876054 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.876145 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.876505 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.876510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.876980 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.877163 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.877175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.878602 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.878606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.879921 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.879936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.880856 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.880861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.881016 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.881688 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.881705 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.884258 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.887994 (+ 3736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.887998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.888000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.888580 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.888991 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.888994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.888997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.889130 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.889824 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.889827 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.889969 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.890090 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.890313 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 06:53:44.890863 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.875801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.18.39.125:46625)
0619 06:53:44.876008 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.876015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.876125 (+ 110us) server_negotiation.cc:407] Connection header received
0619 06:53:44.876248 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.876254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.876335 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.876445 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.877354 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.878449 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.880062 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.880721 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.880863 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.884490 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.884500 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.884506 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.884559 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.887850 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.888682 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.888689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.888695 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.888871 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.889264 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.889272 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.889278 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.889697 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.889859 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.890256 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.890479 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:53:44.891887 32254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:53:44.894484 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.877026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43936 (local address 127.18.39.126:33283)
0619 06:53:44.877196 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.877203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.877221 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:53:44.877423 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.877427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.877485 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.877553 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.878982 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.880089 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.881910 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.882559 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.882699 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.887567 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.887577 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.887583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.887628 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.891197 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.892168 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.892173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.892175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.892296 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.892788 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.892793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.892796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.893192 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.893493 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.893947 (+ 454us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.894171 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:53:44.907764 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.892628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49944)
0619 06:53:44.892889 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.892904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.892985 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.893265 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.893269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.893665 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.893800 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.893808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.895165 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.895173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.896462 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.896475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.897488 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.897493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.897636 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.899107 (+ 1471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.899125 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.901148 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.904922 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.904930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.904934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.905501 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.905963 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.905967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.905970 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.906155 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.906812 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.906815 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.907023 (+ 208us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.907181 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.907477 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183}
I20260619 06:53:44.908069 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.892643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.18.39.124:46755)
0619 06:53:44.892825 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.892832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.892968 (+ 136us) server_negotiation.cc:407] Connection header received
0619 06:53:44.893060 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.893065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.893137 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.893241 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.893926 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.895048 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.896591 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.897307 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.897494 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.901416 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.901439 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.901446 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.901485 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.904787 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.905672 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.905678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.905682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.905830 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.906302 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.906308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.906311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.906684 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.906907 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.907322 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.907687 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":51}
I20260619 06:53:44.912700 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.896144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.18.39.125:46625)
0619 06:53:44.896324 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.896330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.896349 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:53:44.896476 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.896482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.896565 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.896651 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.898211 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.898914 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.900154 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.900597 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.900746 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.906239 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.906249 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.906253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.906301 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.910027 (+ 3726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.910916 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.910920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.910922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.911049 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.911441 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.911444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.911446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.911713 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.911926 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.912258 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.912447 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:53:44.914179 32254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: Bootstrap starting.
I20260619 06:53:44.917932 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:53:44.918012 32254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:44.919078 32254 log.cc:824] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:44.922178 32254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: No bootstrap required, opened a new log
I20260619 06:53:44.938797 32254 raft_consensus.cc:348] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:44.939399 32254 raft_consensus.cc:374] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:44.939574 32254 raft_consensus.cc:729] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e85eec1a89f3452194b9d26463379cc2, State: Initialized, Role: FOLLOWER
I20260619 06:53:44.940374 32254 consensus_queue.cc:260] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:44.955665 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.924076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.18.39.126:33283)
0619 06:53:44.924305 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.924313 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.935054 (+ 10741us) server_negotiation.cc:407] Connection header received
0619 06:53:44.935719 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.935725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.935830 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.935933 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.939326 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.940537 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.943021 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.943661 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.943767 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.948235 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.948244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.948249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.948295 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.952137 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.953142 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.953149 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.953153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.953326 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.953821 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.953828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.953834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.954221 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.954457 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.955012 (+ 555us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.955295 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 06:53:44.955892 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.923929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48092 (local address 127.18.39.125:46625)
0619 06:53:44.928826 (+ 4897us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.928835 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.938204 (+ 9369us) server_negotiation.cc:407] Connection header received
0619 06:53:44.938531 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.938538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.938633 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.938741 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.941009 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.942193 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.944371 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.945076 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.945176 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.948978 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.948985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.948989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.949038 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.952681 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.953596 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.953602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.953605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.953757 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.954166 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.954171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.954176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.954555 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.954765 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.955318 (+ 553us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.955546 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4719}
I20260619 06:53:44.957964 32106 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 4
I20260619 06:53:44.960359 32267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:44.960835 32267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:44.961912 32254 sys_catalog.cc:564] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:44.962781 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.923383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.18.39.124:46755)
0619 06:53:44.923863 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.923876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.938484 (+ 14608us) server_negotiation.cc:407] Connection header received
0619 06:53:44.938676 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.938682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.938777 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.938873 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.942014 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.943538 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.946549 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.947526 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.947652 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.953252 (+ 5600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.953263 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.953271 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.953329 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.958267 (+ 4938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.959409 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.959419 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.959425 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.959672 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.960141 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.960148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.960153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.960638 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.960901 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.962113 (+ 1212us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.962361 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322}
Jun 19 06:53:44 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852023, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:53:44.976243 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.958489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49968)
0619 06:53:44.958674 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:44.958695 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:44.958808 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:44.959994 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:44.959999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:44.960478 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:44.960684 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.960699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.962685 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.962693 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.964527 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:44.964550 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.965602 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.965608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:44.965815 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.966525 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:44.966550 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:44.969186 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:44.973130 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.973136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.973140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.973782 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.974203 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:44.974211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:44.974215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:44.974398 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:44.975188 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:44.975193 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:44.975404 (+ 211us) client_negotiation.cc:769] Sending connection context
0619 06:53:44.975556 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 06:53:44.975855 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 06:53:44.976351 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:44.958884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.18.39.124:46755)
0619 06:53:44.959608 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:44.959616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:44.959639 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:53:44.959694 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:44.959699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:44.959804 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:44.960950 (+ 1146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:44.961229 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.962534 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.964701 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:44.965434 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:44.965590 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:44.969445 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:44.969454 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:44.969461 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:44.969509 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:44.972994 (+ 3485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.973898 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.973904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.973907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.974058 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:44.974544 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:44.974548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:44.974551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:44.974957 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:44.975090 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:44.975657 (+ 567us) server_negotiation.cc:299] Negotiation successful
0619 06:53:44.975901 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"spinlock_wait_cycles":22272,"thread_start_us":540,"threads_started":1}
W20260619 06:53:44.978123 32279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:53:44.978341 32279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:53:44.982328 32106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: Bootstrap starting.
I20260619 06:53:44.973564 32187 master_runner.cc:186] Error getting master registration for 127.18.39.126:33283: OK, instance_id { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" instance_seqno: 1781852023811085 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:53:44.985122 32187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:53:44.987128 32106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:44.988209 32106 log.cc:824] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:44.991219 32106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: No bootstrap required, opened a new log
I20260619 06:53:45.000533 32106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.001034 32106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:45.001188 32106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 718fcdf5d64e487a9ab74b19b2b7c0ad, State: Initialized, Role: FOLLOWER
I20260619 06:53:45.001863 32106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.002866 32282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.003254 32282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.004086 32106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:53:45.019672 32293 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:53:45.019827 32293 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:53:45.019187 32039 master_runner.cc:186] Error getting master registration for 127.18.39.125:46625: OK, instance_id { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" instance_seqno: 1781852024314693 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:53:45.028578 32039 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 06:53:45.049962 32060 tablet.cc:2369] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:45.059988 32180 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } attempt: 5
Jun 19 06:53:45 dist-test-slave-1h5s krb5kdc[32026](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852024, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
W20260619 06:53:45.073923 32208 tablet.cc:2369] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:45.077353 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:45.060781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.18.39.124:46755)
0619 06:53:45.060990 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:45.060996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:45.061053 (+ 57us) server_negotiation.cc:407] Connection header received
0619 06:53:45.061257 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:45.061263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:45.061382 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:45.061497 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:45.062405 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:45.063283 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:45.064548 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:45.065017 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:45.065154 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:45.069838 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:45.069843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:45.069847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:45.069878 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:45.072835 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:45.073619 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:45.073624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:45.073627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:45.073804 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:45.075283 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:45.075291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:45.075297 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:45.075755 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:45.076124 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:45.076737 (+ 613us) server_negotiation.cc:299] Negotiation successful
0619 06:53:45.076962 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:53:45.077376 32182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:45.060676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:46755 (local address 127.0.0.1:49970)
0619 06:53:45.060966 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:45.060989 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:45.061126 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:45.061536 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:45.061541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:45.062066 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:45.062252 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:45.062261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:45.063411 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:45.063417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:45.064401 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:45.064413 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:45.065181 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:45.065186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:45.065421 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:45.066878 (+ 1457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:45.066905 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:45.069641 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:45.073002 (+ 3361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:45.073007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:45.073011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:45.073504 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:45.074902 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:45.074906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:45.074909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:45.075064 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:45.076080 (+ 1016us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:45.076085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:45.076364 (+ 279us) client_negotiation.cc:769] Sending connection context
0619 06:53:45.076603 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 06:53:45.076929 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260619 06:53:45.082613 32180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: Bootstrap starting.
I20260619 06:53:45.087262 32180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:45.088337 32180 log.cc:824] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:45.091307 32180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: No bootstrap required, opened a new log
I20260619 06:53:45.098507 32180 raft_consensus.cc:348] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.098866 32180 raft_consensus.cc:374] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:45.099009 32180 raft_consensus.cc:729] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e80c298f60e64883a810e54506e7b02d, State: Initialized, Role: FOLLOWER
I20260619 06:53:45.099676 32180 consensus_queue.cc:260] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.102968 32180 sys_catalog.cc:564] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:45.104103 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.104791 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.129753 32113 master_runner.cc:263] No leader master found from master e80c298f60e64883a810e54506e7b02d
I20260619 06:53:45.129941 32113 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 06:53:45.135632 32307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:53:45.135818 32307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:53:45.287994 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:45.288338 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.294806 32267 leader_election.cc:288] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 718fcdf5d64e487a9ab74b19b2b7c0ad (127.18.39.126:33283), e80c298f60e64883a810e54506e7b02d (127.18.39.125:46625)
I20260619 06:53:45.296213 32155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e85eec1a89f3452194b9d26463379cc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80c298f60e64883a810e54506e7b02d" is_pre_election: true
I20260619 06:53:45.296916 32155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e85eec1a89f3452194b9d26463379cc2 in term 0.
I20260619 06:53:45.301014 32282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:45.301311 32282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.301713 32204 leader_election.cc:302] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e80c298f60e64883a810e54506e7b02d, e85eec1a89f3452194b9d26463379cc2; no voters:
I20260619 06:53:45.302397 32267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:53:45.302560 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:53:45.302999 32282 leader_election.cc:288] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e80c298f60e64883a810e54506e7b02d (127.18.39.125:46625), e85eec1a89f3452194b9d26463379cc2 (127.18.39.124:46755)
I20260619 06:53:45.304010 32081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e85eec1a89f3452194b9d26463379cc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" is_pre_election: true
W20260619 06:53:45.304530 32134 tablet.cc:2369] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:45.304594 32081 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e85eec1a89f3452194b9d26463379cc2 in term 0.
I20260619 06:53:45.309527 32155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80c298f60e64883a810e54506e7b02d" is_pre_election: true
I20260619 06:53:45.309805 32155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 718fcdf5d64e487a9ab74b19b2b7c0ad in term 0.
I20260619 06:53:45.310446 32056 leader_election.cc:302] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 718fcdf5d64e487a9ab74b19b2b7c0ad, e80c298f60e64883a810e54506e7b02d; no voters:
I20260619 06:53:45.310984 32282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:53:45.311138 32282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:53:45.311240 32282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:45.314941 32282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.316546 32155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80c298f60e64883a810e54506e7b02d"
I20260619 06:53:45.316798 32155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:45.323037 32155 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 718fcdf5d64e487a9ab74b19b2b7c0ad in term 1.
I20260619 06:53:45.325420 32267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:45.334400 32229 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e85eec1a89f3452194b9d26463379cc2" is_pre_election: true
I20260619 06:53:45.339010 32228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e85eec1a89f3452194b9d26463379cc2"
I20260619 06:53:45.338330 32282 leader_election.cc:288] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [CANDIDATE]: Term 1 election: Requested vote from peers e80c298f60e64883a810e54506e7b02d (127.18.39.125:46625), e85eec1a89f3452194b9d26463379cc2 (127.18.39.124:46755)
I20260619 06:53:45.351823 32056 leader_election.cc:302] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 718fcdf5d64e487a9ab74b19b2b7c0ad, e80c298f60e64883a810e54506e7b02d; no voters: e85eec1a89f3452194b9d26463379cc2
I20260619 06:53:45.352829 32282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:45.354122 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.355598 32081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e85eec1a89f3452194b9d26463379cc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad"
I20260619 06:53:45.358055 32229 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 718fcdf5d64e487a9ab74b19b2b7c0ad in current term 1: Already voted for candidate e85eec1a89f3452194b9d26463379cc2 in this term.
I20260619 06:53:45.360672 32155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e85eec1a89f3452194b9d26463379cc2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e80c298f60e64883a810e54506e7b02d"
I20260619 06:53:45.361169 32155 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e85eec1a89f3452194b9d26463379cc2 in current term 1: Already voted for candidate 718fcdf5d64e487a9ab74b19b2b7c0ad in this term.
I20260619 06:53:45.362365 32267 leader_election.cc:288] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [CANDIDATE]: Term 1 election: Requested vote from peers 718fcdf5d64e487a9ab74b19b2b7c0ad (127.18.39.126:33283), e80c298f60e64883a810e54506e7b02d (127.18.39.125:46625)
I20260619 06:53:45.371277 32282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [term 1 LEADER]: Becoming Leader. State: Replica: 718fcdf5d64e487a9ab74b19b2b7c0ad, State: Running, Role: LEADER
I20260619 06:53:45.372290 32282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } }
I20260619 06:53:45.393172 32205 leader_election.cc:302] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e85eec1a89f3452194b9d26463379cc2; no voters: 718fcdf5d64e487a9ab74b19b2b7c0ad, e80c298f60e64883a810e54506e7b02d
I20260619 06:53:45.393988 32267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260619 06:53:45.394627 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718fcdf5d64e487a9ab74b19b2b7c0ad. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.395200 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:45.406975 32311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:45.409572 32311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:45.439284 32229 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [term 1 FOLLOWER]: Refusing update from remote peer 718fcdf5d64e487a9ab74b19b2b7c0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:45.440178 32309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:45.447770 32155 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [term 1 FOLLOWER]: Refusing update from remote peer 718fcdf5d64e487a9ab74b19b2b7c0ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:45.448612 32309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:45.462672 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718fcdf5d64e487a9ab74b19b2b7c0ad. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.463006 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.465459 32312 mvcc.cc:205] Tried to move back new op lower bound from 7298465896187453440 to 7298465895931265024. Current Snapshot: MvccSnapshot[applied={T|T < 7298465896187453440}]
I20260619 06:53:45.465983 32309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.466230 32309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:45.466563 32315 mvcc.cc:205] Tried to move back new op lower bound from 7298465896187453440 to 7298465895931265024. Current Snapshot: MvccSnapshot[applied={T|T < 7298465896187453440}]
I20260619 06:53:45.468108 32311 catalog_manager.cc:1347] Generated new cluster ID: 645fab5444d247c39cba49711e1c66e5
I20260619 06:53:45.468209 32311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:45.470012 32267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 718fcdf5d64e487a9ab74b19b2b7c0ad. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.470278 32267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.472589 32282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.472966 32282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:45.474514 32313 mvcc.cc:205] Tried to move back new op lower bound from 7298465896187453440 to 7298465895931265024. Current Snapshot: MvccSnapshot[applied={T|T < 7298465896187453440}]
I20260619 06:53:45.483225 32267 sys_catalog.cc:455] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.483515 32267 sys_catalog.cc:458] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.483901 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "718fcdf5d64e487a9ab74b19b2b7c0ad" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 33283 } } peers { permanent_uuid: "e80c298f60e64883a810e54506e7b02d" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 46625 } } peers { permanent_uuid: "e85eec1a89f3452194b9d26463379cc2" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 46755 } } }
I20260619 06:53:45.484158 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:53:45.501145 32311 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:53:45.503242 32311 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:45.525207 32311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 718fcdf5d64e487a9ab74b19b2b7c0ad: Generated new TSK 0
I20260619 06:53:45.526441 32311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:53:45.980553 32279 catalog_manager.cc:1259] Loaded cluster ID: 645fab5444d247c39cba49711e1c66e5
I20260619 06:53:45.980659 32279 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: loading cluster ID for follower catalog manager: success
I20260619 06:53:45.983357 32279 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: acquiring CA information for follower catalog manager: success
I20260619 06:53:45.985513 32279 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e85eec1a89f3452194b9d26463379cc2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:53:46.138471 32307 catalog_manager.cc:1259] Loaded cluster ID: 645fab5444d247c39cba49711e1c66e5
I20260619 06:53:46.138593 32307 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: loading cluster ID for follower catalog manager: success
I20260619 06:53:46.141472 32307 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: acquiring CA information for follower catalog manager: success
I20260619 06:53:46.143508 32307 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e80c298f60e64883a810e54506e7b02d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:53:46.655948 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32039
I20260619 06:53:46.657785 32101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:46.770061 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32039
I20260619 06:53:46.790617 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32113
I20260619 06:53:46.791877 32175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:46.910925 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32113
I20260619 06:53:46.930881 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32187
I20260619 06:53:46.932077 32249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:47.042019 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32187
2026-06-19T06:53:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5032 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:53:47 dist-test-slave-1h5s krb5kdc[32331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:53:47 dist-test-slave-1h5s krb5kdc[32331](info): set up 2 sockets
Jun 19 06:53:47 dist-test-slave-1h5s krb5kdc[32331](info): commencing operation
krb5kdc: starting...
W20260619 06:53:48.149870 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:53:48 dist-test-slave-1h5s krb5kdc[32331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:53:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:53:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:53:48.392529 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35667
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:48.732021 32344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:48.732337 32344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:48.732504 32344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:48.740082 32344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:48.740216 32344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:48.740332 32344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:48.740454 32344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:48.740537 32344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:48.750952 32344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:48.752413 32344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:48.753903 32344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:48.766256 32349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:48.766171 32352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:48.766197 32350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:48.767458 32344 server_base.cc:1034] running on GCE node
I20260619 06:53:48.768280 32344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:48.769848 32344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:48.771085 32344 hybrid_clock.cc:648] HybridClock initialized: now 1781852028771035 us; error 43 us; skew 500 ppm
I20260619 06:53:48.771440 32344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:48 dist-test-slave-1h5s krb5kdc[32331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852028, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:48.777487 32344 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:48.779182 32344 webserver.cc:460] Webserver started at http://127.18.39.126:43755/ using document root <none> and password file <none>
I20260619 06:53:48.779909 32344 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:48.780061 32344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:48.780371 32344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:48.783468 32344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "e4559ef7d7744533999272313954084f"
format_stamp: "Formatted at 2026-06-19 06:53:48 on dist-test-slave-1h5s"
server_key: "e4aac14b02ed7d92a7c2ee14347f26bf"
server_key_iv: "361f8b3faf268b1208505379a328b48c"
server_key_version: "encryptionkey@0"
I20260619 06:53:48.784334 32344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "e4559ef7d7744533999272313954084f"
format_stamp: "Formatted at 2026-06-19 06:53:48 on dist-test-slave-1h5s"
server_key: "e4aac14b02ed7d92a7c2ee14347f26bf"
server_key_iv: "361f8b3faf268b1208505379a328b48c"
server_key_version: "encryptionkey@0"
I20260619 06:53:48.789991 32344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:53:48.794471 32359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:48.796051 32344 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:53:48.796264 32344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "e4559ef7d7744533999272313954084f"
format_stamp: "Formatted at 2026-06-19 06:53:48 on dist-test-slave-1h5s"
server_key: "e4aac14b02ed7d92a7c2ee14347f26bf"
server_key_iv: "361f8b3faf268b1208505379a328b48c"
server_key_version: "encryptionkey@0"
I20260619 06:53:48.796495 32344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:48.808936 32344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:48.813253 32344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:48.813582 32344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:48.832540 32344 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35667
I20260619 06:53:48.832562 32410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35667 every 8 connection(s)
I20260619 06:53:48.834092 32344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:48.835536 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32344
I20260619 06:53:48.835772 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:53:48.836649 18589 external_mini_cluster.cc:1428] Setting key ce80eb6128c757b88de8c43e1e550c95
I20260619 06:53:48.839021 32411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:53:48 dist-test-slave-1h5s krb5kdc[32331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:48.853031 32411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f: Bootstrap starting.
I20260619 06:53:48.857651 32411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:48.859021 32411 log.cc:824] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:48.862648 32414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:48.838579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.18.39.126:35667)
0619 06:53:48.839396 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:48.839408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:48.839448 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:53:48.841084 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:48.841106 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:48.841593 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:48.841934 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:48.843933 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:48.845307 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:48.847257 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:48.848096 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:48.848234 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:48.853240 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:48.853251 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:48.853272 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:48.853371 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:48.857412 (+ 4041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:48.858311 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:48.858329 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:48.858338 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:48.858568 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:48.858962 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:48.858968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:48.858971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:48.859900 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:48.860139 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:48.860814 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 06:53:48.861271 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":369,"threads_started":1}
I20260619 06:53:48.862887 32411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f: No bootstrap required, opened a new log
I20260619 06:53:48.866935 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:53:48.871394 32411 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } }
I20260619 06:53:48.871836 32411 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:48.871946 32411 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4559ef7d7744533999272313954084f, State: Initialized, Role: FOLLOWER
I20260619 06:53:48.872566 32411 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } }
I20260619 06:53:48.872821 32411 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:48.872967 32411 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:48.873153 32411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:48.875453 32411 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } }
I20260619 06:53:48.875994 32411 leader_election.cc:302] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4559ef7d7744533999272313954084f; no voters:
I20260619 06:53:48.876622 32411 leader_election.cc:288] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:53:48.876828 32416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:48.877563 32416 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [term 1 LEADER]: Becoming Leader. State: Replica: e4559ef7d7744533999272313954084f, State: Running, Role: LEADER
I20260619 06:53:48.878150 32416 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } }
I20260619 06:53:48.878737 32411 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:48.881225 32417 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4559ef7d7744533999272313954084f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } } }
I20260619 06:53:48.881696 32417 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:48.881199 32418 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4559ef7d7744533999272313954084f. Latest consensus state: current_term: 1 leader_uuid: "e4559ef7d7744533999272313954084f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4559ef7d7744533999272313954084f" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35667 } } }
I20260619 06:53:48.881810 32418 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:48.882845 32423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:48.888736 32423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:48.897650 32423 catalog_manager.cc:1347] Generated new cluster ID: 571558f5cb3843bc88342554e0425d02
I20260619 06:53:48.897783 32423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:48.911372 32423 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:53:48.913324 32423 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:48.923280 32423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4559ef7d7744533999272313954084f: Generated new TSK 0
I20260619 06:53:48.924216 32423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:53:48.979391 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32344
I20260619 06:53:48.980700 32406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:49.079833 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32344
2026-06-19T06:53:49Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2032 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:53:49 dist-test-slave-1h5s krb5kdc[32439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:53:49 dist-test-slave-1h5s krb5kdc[32439](info): set up 2 sockets
Jun 19 06:53:49 dist-test-slave-1h5s krb5kdc[32439](info): commencing operation
krb5kdc: starting...
W20260619 06:53:50.177956 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:53:50 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:53:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:53:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:53:50.421069 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:50.761430 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:50.761830 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:50.761993 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:50.769558 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:50.769717 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:50.769836 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:50.769935 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:50.770020 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:50.780386 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:50.781941 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:50.783295 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:50.795814 32457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:50.795919 32458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:50.795853 32460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:50.803141 32452 server_base.cc:1034] running on GCE node
I20260619 06:53:50.804109 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:50.805737 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:50.806999 32452 hybrid_clock.cc:648] HybridClock initialized: now 1781852030806957 us; error 34 us; skew 500 ppm
I20260619 06:53:50.807384 32452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:50 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:50.810757 32452 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:50.812539 32452 webserver.cc:460] Webserver started at http://127.18.39.126:34521/ using document root <none> and password file <none>
I20260619 06:53:50.813288 32452 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:50.813457 32452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:50.813740 32452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:50.816748 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "77ba538586884539b1fb48fb67a35ee8"
format_stamp: "Formatted at 2026-06-19 06:53:50 on dist-test-slave-1h5s"
server_key: "2618a22064c85300e1e6944f355e723a"
server_key_iv: "1b593c787b700c2bea4e268025908e64"
server_key_version: "encryptionkey@0"
I20260619 06:53:50.817569 32452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "77ba538586884539b1fb48fb67a35ee8"
format_stamp: "Formatted at 2026-06-19 06:53:50 on dist-test-slave-1h5s"
server_key: "2618a22064c85300e1e6944f355e723a"
server_key_iv: "1b593c787b700c2bea4e268025908e64"
server_key_version: "encryptionkey@0"
I20260619 06:53:50.823112 32452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:53:50.826776 32467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:50.828194 32452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:53:50.828420 32452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "77ba538586884539b1fb48fb67a35ee8"
format_stamp: "Formatted at 2026-06-19 06:53:50 on dist-test-slave-1h5s"
server_key: "2618a22064c85300e1e6944f355e723a"
server_key_iv: "1b593c787b700c2bea4e268025908e64"
server_key_version: "encryptionkey@0"
I20260619 06:53:50.828641 32452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:50.848043 32452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:50.852993 32452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:50.853303 32452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:50.872051 32452 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:41087
I20260619 06:53:50.872076 32518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:41087 every 8 connection(s)
I20260619 06:53:50.873548 32452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:50.874749 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32452
I20260619 06:53:50.874975 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:53:50.875627 18589 external_mini_cluster.cc:1428] Setting key 0c32880a4ee2792acbccbe651f745810
I20260619 06:53:50.877964 32519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:50.891937 32519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap starting.
Jun 19 06:53:50 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:50.895658 32519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:50.896761 32519 log.cc:824] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:50.900243 32519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: No bootstrap required, opened a new log
I20260619 06:53:50.904078 32522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:50.877780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.18.39.126:41087)
0619 06:53:50.881936 (+ 4156us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:50.881949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:50.881996 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:53:50.883643 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:50.883667 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:50.884116 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:50.884490 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:50.886346 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:50.887661 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:50.889583 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:50.890312 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:50.890459 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:50.895374 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:50.895383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:50.895400 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:50.895444 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:50.899297 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:50.900089 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:50.900099 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:50.900107 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:50.900327 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:50.900718 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:50.900724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:50.900729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:50.901497 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:50.901829 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:50.902151 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 06:53:50.902637 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":3909,"thread_start_us":3703,"threads_started":1}
I20260619 06:53:50.909298 32519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:50.909726 32519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:50.909801 32519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Initialized, Role: FOLLOWER
I20260619 06:53:50.910290 32519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:50.910490 32519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:50.910588 32519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:50.910715 32519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:50.912873 32519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:50.913511 32519 leader_election.cc:302] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77ba538586884539b1fb48fb67a35ee8; no voters:
I20260619 06:53:50.914142 32519 leader_election.cc:288] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:53:50.914324 32524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:50.915119 32524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 LEADER]: Becoming Leader. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Running, Role: LEADER
I20260619 06:53:50.915670 32524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:50.916440 32519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:50.918349 32525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:50.918732 32525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:50.918936 32526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77ba538586884539b1fb48fb67a35ee8. Latest consensus state: current_term: 1 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:50.919333 32526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:50.927378 32540 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:53:50.927630 32529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:50.932042 32529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:50.941144 32529 catalog_manager.cc:1347] Generated new cluster ID: 94dd0ba78a664d8991e222d2ddee9ede
I20260619 06:53:50.941272 32529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:50.968551 32529 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:53:50.970063 32529 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:50.979188 32529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Generated new TSK 0
I20260619 06:53:50.980294 32529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:53:51.036934 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:51.370563 32545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:51.370855 32545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:51.371022 32545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:51.378680 32545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:51.378808 32545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:51.379006 32545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:53:51.389472 32545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:51.390913 32545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:51.392313 32545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:51.405953 32550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:51.405929 32551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:51.407570 32545 server_base.cc:1034] running on GCE node
W20260619 06:53:51.408526 32553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:51.409268 32545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:51.410333 32545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:51.411576 32545 hybrid_clock.cc:648] HybridClock initialized: now 1781852031411540 us; error 44 us; skew 500 ppm
I20260619 06:53:51.411940 32545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:51 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852031, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:51.415287 32545 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:53:51.416985 32545 webserver.cc:460] Webserver started at http://127.18.39.65:35765/ using document root <none> and password file <none>
I20260619 06:53:51.417860 32545 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:51.417999 32545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:51.418356 32545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:51.421463 32545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "16d18d485ca8417b8a587c341707f872"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "c0a09219bf4442ba6df0284eea66e8e1"
server_key_iv: "6b7a8af2af85ad2e695468571932f363"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.422367 32545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "16d18d485ca8417b8a587c341707f872"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "c0a09219bf4442ba6df0284eea66e8e1"
server_key_iv: "6b7a8af2af85ad2e695468571932f363"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.428073 32545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:53:51.431988 32560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:51.433820 32545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:53:51.434015 32545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "16d18d485ca8417b8a587c341707f872"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "c0a09219bf4442ba6df0284eea66e8e1"
server_key_iv: "6b7a8af2af85ad2e695468571932f363"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.434235 32545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:51.473088 32545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:51.477487 32545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:51.477813 32545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:51.478797 32545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:51.480283 32545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:51.480394 32545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:51.480543 32545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:51.480613 32545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:51.514050 32545 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42631
I20260619 06:53:51.514135 32672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42631 every 8 connection(s)
I20260619 06:53:51.515393 32545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:53:51.520350 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32545
I20260619 06:53:51.520565 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:53:51.521271 18589 external_mini_cluster.cc:1428] Setting key ea8ab833956e689047da0264c04cc2cb
Jun 19 06:53:51 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852031, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:51.537907 32675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:51.518126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:51149 (local address 127.18.39.126:41087)
0619 06:53:51.518596 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:51.518600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:51.519411 (+ 811us) server_negotiation.cc:407] Connection header received
0619 06:53:51.521392 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:51.521399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:51.521500 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:51.521622 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:51.523300 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:51.524064 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:51.525759 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:51.526174 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:51.526319 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:51.530270 (+ 3951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:51.530277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:51.530281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:51.530319 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:51.534158 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:51.535051 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:51.535060 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:51.535066 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:51.535227 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:51.535739 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:51.535747 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:51.535754 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:51.536171 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:51.536334 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:51.537078 (+ 744us) server_negotiation.cc:299] Negotiation successful
0619 06:53:51.537385 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":262,"threads_started":1}
I20260619 06:53:51.538988 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:51.518568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:51149)
0619 06:53:51.519249 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:51.519286 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:51.521086 (+ 1800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:51.521885 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:51.521893 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:51.522436 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:51.523126 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:51.523153 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:51.524220 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:51.524225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:51.525594 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:51.525607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:51.526409 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:51.526416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:51.526683 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:51.527422 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:51.527446 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:51.529981 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:51.534351 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:51.534361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:51.534379 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:51.534909 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:51.535391 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:51.535394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:51.535397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:51.535615 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:51.536301 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:51.536308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:51.536669 (+ 361us) client_negotiation.cc:769] Sending connection context
0619 06:53:51.536916 (+ 247us) client_negotiation.cc:240] Negotiation successful
0619 06:53:51.537459 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":281,"threads_started":1}
I20260619 06:53:51.540723 32673 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
I20260619 06:53:51.541071 32673 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:51.541900 32673 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
I20260619 06:53:51.544435 32484 ts_manager.cc:194] Registered new tserver with Master: 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:53:51.546205 32484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:51149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:53:51.574285 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:51.907793 32679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:51.908088 32679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:51.908219 32679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:51.915740 32679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:51.915900 32679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:51.916098 32679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:53:51.926522 32679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:51.927886 32679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:51.929230 32679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:51.943537 32684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:51.943500 32687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:51.943572 32685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:51.944788 32679 server_base.cc:1034] running on GCE node
I20260619 06:53:51.945582 32679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:51.946630 32679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:51.947875 32679 hybrid_clock.cc:648] HybridClock initialized: now 1781852031947809 us; error 57 us; skew 500 ppm
I20260619 06:53:51.948225 32679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:51 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852031, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:51.951581 32679 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:53:51.953079 32679 webserver.cc:460] Webserver started at http://127.18.39.66:39933/ using document root <none> and password file <none>
I20260619 06:53:51.953840 32679 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:51.953949 32679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:51.954193 32679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:51.957218 32679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "5d8d9be338e8498b845bad51c017ec31"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "1f03b2ccf9a7829737d53fd070363219"
server_key_iv: "9a6db13588fcfd5d77f4f987862c9f8d"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.958076 32679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "5d8d9be338e8498b845bad51c017ec31"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "1f03b2ccf9a7829737d53fd070363219"
server_key_iv: "9a6db13588fcfd5d77f4f987862c9f8d"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.963402 32679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:53:51.967217 32694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:51.968827 32679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:53:51.969022 32679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "5d8d9be338e8498b845bad51c017ec31"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "1f03b2ccf9a7829737d53fd070363219"
server_key_iv: "9a6db13588fcfd5d77f4f987862c9f8d"
server_key_version: "encryptionkey@0"
I20260619 06:53:51.969267 32679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:52.012676 32679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:52.017547 32679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:52.017913 32679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:52.018939 32679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:52.020424 32679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:52.020498 32679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:52.020643 32679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:52.020704 32679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:52.055171 32679 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:41747
I20260619 06:53:52.055226 338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:41747 every 8 connection(s)
I20260619 06:53:52.056511 32679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:53:52.058704 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 32679
I20260619 06:53:52.058911 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:53:52.059497 18589 external_mini_cluster.cc:1428] Setting key 352998e6d38da8bd1dff15fa5a1c1833
Jun 19 06:53:52 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852031, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:52.080196 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.059318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:55219 (local address 127.18.39.126:41087)
0619 06:53:52.059852 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.059858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.060856 (+ 998us) server_negotiation.cc:407] Connection header received
0619 06:53:52.062864 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.062873 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.062981 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.063106 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:52.064646 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.065895 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.067480 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.068281 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.068433 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.073479 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:52.073485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:52.073489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:52.073526 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:52.076460 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.077581 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.077586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.077588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.077731 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.078283 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.078286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.078288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.078572 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:52.078684 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.079690 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.079928 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260619 06:53:52.081929 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.059842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:55219)
0619 06:53:52.060683 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.060738 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.062544 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.063313 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.063322 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.063888 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:52.064460 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.064476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.066051 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.066055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.067329 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.067341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.068511 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.068516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.068935 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.069809 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:52.069857 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:52.073055 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:52.076663 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:52.076677 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:52.076700 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:52.077445 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:52.077870 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:52.077878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:52.077885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:52.078161 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:52.078713 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:52.078724 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:52.079228 (+ 504us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.079580 (+ 352us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.080222 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":362,"threads_started":1}
I20260619 06:53:52.083684 339 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:53:52.084071 339 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:52.084916 339 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
I20260619 06:53:52.086719 32484 ts_manager.cc:194] Registered new tserver with Master: 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
I20260619 06:53:52.087967 32484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:55219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:53:52.115037 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:52.452584 345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:52.452924 345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:52.453099 345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:52.460520 345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:52.460702 345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:52.460903 345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:53:52.471129 345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:52.472540 345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:52.474098 345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:52.487555 351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:52.487793 350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:52.488024 353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:52.488435 345 server_base.cc:1034] running on GCE node
I20260619 06:53:52.489192 345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:52.490226 345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:52.491479 345 hybrid_clock.cc:648] HybridClock initialized: now 1781852032491441 us; error 61 us; skew 500 ppm
I20260619 06:53:52.491870 345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:52 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852032, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:52.495594 345 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:53:52.497128 345 webserver.cc:460] Webserver started at http://127.18.39.67:43083/ using document root <none> and password file <none>
I20260619 06:53:52.497895 345 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:52.498030 345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:52.498355 345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:53:52.501279 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
format_stamp: "Formatted at 2026-06-19 06:53:52 on dist-test-slave-1h5s"
server_key: "b10b08fb0df403aa447a2d789c7d87cc"
server_key_iv: "6f4206d2bfed8aaff2e96c641d1844b4"
server_key_version: "encryptionkey@0"
I20260619 06:53:52.502139 345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
format_stamp: "Formatted at 2026-06-19 06:53:52 on dist-test-slave-1h5s"
server_key: "b10b08fb0df403aa447a2d789c7d87cc"
server_key_iv: "6f4206d2bfed8aaff2e96c641d1844b4"
server_key_version: "encryptionkey@0"
I20260619 06:53:52.507282 345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:53:52.510826 360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:52.512121 345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:53:52.512302 345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
format_stamp: "Formatted at 2026-06-19 06:53:52 on dist-test-slave-1h5s"
server_key: "b10b08fb0df403aa447a2d789c7d87cc"
server_key_iv: "6f4206d2bfed8aaff2e96c641d1844b4"
server_key_version: "encryptionkey@0"
I20260619 06:53:52.512508 345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:52.529211 345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:52.533921 345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:52.534220 345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:52.535190 345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:52.536561 345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:52.536664 345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:52.536823 345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:52.536899 345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:52.549557 32673 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:52.570960 345 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44775
I20260619 06:53:52.571053 472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44775 every 8 connection(s)
I20260619 06:53:52.572409 345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:53:52.578450 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 345
I20260619 06:53:52.578658 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:53:52.579124 18589 external_mini_cluster.cc:1428] Setting key 9b2122d127de29806e500752b657ade6
Jun 19 06:53:52 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852032, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:52.592985 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.575165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:48251 (local address 127.18.39.126:41087)
0619 06:53:52.575444 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.575449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.576466 (+ 1017us) server_negotiation.cc:407] Connection header received
0619 06:53:52.578420 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.578427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.578528 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.578652 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:52.580568 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.581731 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.583723 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.584128 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.584224 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.587693 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:52.587698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:52.587701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:52.587733 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:52.590170 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.590882 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.590886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.590889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.591015 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.591389 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.591392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.591394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.591662 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:52.591797 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.592449 (+ 652us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.592629 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260619 06:53:52.594352 475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.575604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:48251)
0619 06:53:52.576299 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.576346 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.578135 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.578948 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.578957 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.579516 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:52.580316 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.580344 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.581895 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.581900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.583592 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.583605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.584310 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.584315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.584529 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.585230 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:52.585252 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:52.587485 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:52.590296 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:52.590303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:52.590318 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:52.590749 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:52.591110 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:52.591113 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:52.591115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:52.591299 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:52.591775 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:52.591780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:52.592119 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.592369 (+ 250us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.592801 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":278,"threads_started":1}
I20260619 06:53:52.595948 473 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
I20260619 06:53:52.596290 473 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:52.597138 473 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
I20260619 06:53:52.598740 32484 ts_manager.cc:194] Registered new tserver with Master: eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775)
I20260619 06:53:52.599581 32484 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:48251
I20260619 06:53:52.607758 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:53:52.626231 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.612419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.18.39.126:41087)
0619 06:53:52.612669 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.612676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.612967 (+ 291us) server_negotiation.cc:407] Connection header received
0619 06:53:52.613153 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.613158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.613222 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.613309 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:52.614974 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.615697 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.617492 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.617949 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.618047 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.620724 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:52.620729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:52.620732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:52.620763 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:52.623644 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.624369 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.624372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.624375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.624505 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.624806 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.624809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.624811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.625126 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:52.625320 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.625691 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.625876 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:53:52.630484 32484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:53:52.637642 32484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:53:52.660728 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.651025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:41747 (local address 127.0.0.1:38492)
0619 06:53:52.651577 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.651594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.651705 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.653502 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.653510 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.653548 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:52.654005 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.654017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.655515 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.655523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.657747 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.657760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.659589 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.659596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.659830 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.659892 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.660001 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.660235 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":68,"thread_start_us":294,"threads_started":1}
I20260619 06:53:52.661268 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.651647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33520)
0619 06:53:52.652243 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.652262 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.652382 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.653770 (+ 1388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.653777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.653818 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:52.654243 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.654252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.656046 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.656054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.658414 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.658424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.660292 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.660307 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.660631 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.660665 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.660743 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.660887 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":60,"thread_start_us":844,"threads_started":1}
I20260619 06:53:52.661417 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.651240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.18.39.66:41747)
0619 06:53:52.652876 (+ 1636us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.652884 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.652910 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:53:52.652997 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.653004 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.653249 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.654074 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:52.654268 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.655383 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.657905 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.659358 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.660593 (+ 1235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.660799 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.660878 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.661049 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1480,"thread_start_us":1346,"threads_started":1}
I20260619 06:53:52.661448 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.650116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.18.39.67:44775)
0619 06:53:52.650901 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.650910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.650945 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:53:52.651060 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.651069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.651328 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.651546 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:52.652767 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.654478 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.658299 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.659721 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.659842 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.660085 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.660871 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.661055 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":422,"threads_started":1}
I20260619 06:53:52.662401 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.652289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.18.39.65:42631)
0619 06:53:52.653232 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.653241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.653268 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:53:52.653392 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.653400 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.653606 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.653786 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:52.654409 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.655919 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.658561 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.659721 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.659857 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.660082 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.661888 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.662051 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":776,"thread_start_us":1214,"threads_started":1}
I20260619 06:53:52.663087 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.649843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44775 (local address 127.0.0.1:35332)
0619 06:53:52.650540 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.650587 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.650776 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.651963 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.651974 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.652035 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:52.652551 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.652567 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.654630 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.654634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.658121 (+ 3487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.658136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.660255 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.660265 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.660467 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.660631 (+ 164us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.660732 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.662717 (+ 1985us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":2590,"thread_start_us":318,"threads_started":1}
I20260619 06:53:52.665730 32742 tablet_service.cc:1467] Processing CreateTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa (DEFAULT_TABLE table=test-table [id=06033216387941e892c338b932348636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:52.666139 408 tablet_service.cc:1467] Processing CreateTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa (DEFAULT_TABLE table=test-table [id=06033216387941e892c338b932348636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:52.666137 32608 tablet_service.cc:1467] Processing CreateTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa (DEFAULT_TABLE table=test-table [id=06033216387941e892c338b932348636]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:52.667279 408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b29e26923ee4d67be493ba907f1a6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:52.667289 32742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b29e26923ee4d67be493ba907f1a6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:52.667448 32608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b29e26923ee4d67be493ba907f1a6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:52.679148 491 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Bootstrap starting.
I20260619 06:53:52.683210 491 tablet_bootstrap.cc:654] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:52.684408 491 log.cc:824] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:52.686692 493 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Bootstrap starting.
I20260619 06:53:52.686764 492 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Bootstrap starting.
I20260619 06:53:52.687534 491 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: No bootstrap required, opened a new log
I20260619 06:53:52.687812 491 ts_tablet_manager.cc:1397] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260619 06:53:52.690503 493 tablet_bootstrap.cc:654] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:52.690503 492 tablet_bootstrap.cc:654] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:52.691879 492 log.cc:824] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:52.691879 493 log.cc:824] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:52.695107 492 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: No bootstrap required, opened a new log
I20260619 06:53:52.695410 492 ts_tablet_manager.cc:1397] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 06:53:52.695494 493 tablet_bootstrap.cc:492] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: No bootstrap required, opened a new log
I20260619 06:53:52.695803 493 ts_tablet_manager.cc:1397] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:53:52.697638 491 raft_consensus.cc:348] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.698115 491 raft_consensus.cc:374] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:52.698211 491 raft_consensus.cc:729] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca9a8e0bf5541539222fdb2b9651e7c, State: Initialized, Role: FOLLOWER
I20260619 06:53:52.698848 491 consensus_queue.cc:260] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.700057 473 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:52.700695 491 ts_tablet_manager.cc:1428] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260619 06:53:52.704397 492 raft_consensus.cc:348] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.704879 492 raft_consensus.cc:374] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:52.704988 492 raft_consensus.cc:729] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8d9be338e8498b845bad51c017ec31, State: Initialized, Role: FOLLOWER
I20260619 06:53:52.705751 492 consensus_queue.cc:260] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.705785 493 raft_consensus.cc:348] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.706140 493 raft_consensus.cc:374] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:52.706256 493 raft_consensus.cc:729] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d18d485ca8417b8a587c341707f872, State: Initialized, Role: FOLLOWER
I20260619 06:53:52.706977 339 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:52.706869 493 consensus_queue.cc:260] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.707700 492 ts_tablet_manager.cc:1428] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260619 06:53:52.708629 493 ts_tablet_manager.cc:1428] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260619 06:53:52.717243 497 raft_consensus.cc:482] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:52.717615 497 raft_consensus.cc:504] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.719375 497 leader_election.cc:288] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747), 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
I20260619 06:53:52.726094 475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.719631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:41747 (local address 127.18.39.67:36851)
0619 06:53:52.719825 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.719847 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.719981 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.720548 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.720556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.720590 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:52.720800 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.720815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.722120 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.722124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.724094 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.724109 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.725485 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.725489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.725632 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.725658 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.725744 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.725851 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260619 06:53:52.726199 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.719831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:36851 (local address 127.18.39.66:41747)
0619 06:53:52.720042 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.720049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.720071 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:53:52.720145 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.720150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.720229 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.720349 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:52.720954 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.721963 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.724243 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.725255 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.725457 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.725535 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.725797 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.725934 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:53:52.726938 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.720179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:40897 (local address 127.18.39.65:42631)
0619 06:53:52.720435 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.720440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.720841 (+ 401us) server_negotiation.cc:407] Connection header received
0619 06:53:52.721054 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.721060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.721137 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.721237 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:52.721759 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.722613 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.724802 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.725985 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.726144 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.726215 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.726575 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.726726 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:53:52.727466 32762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b29e26923ee4d67be493ba907f1a6aa" candidate_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8d9be338e8498b845bad51c017ec31" is_pre_election: true
I20260619 06:53:52.727922 32762 raft_consensus.cc:2454] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca9a8e0bf5541539222fdb2b9651e7c in term 0.
I20260619 06:53:52.728309 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.720061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.18.39.67:40897)
0619 06:53:52.720733 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:52.720759 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:52.720918 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:52.721252 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:52.721255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:52.721275 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:52.721618 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.721626 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.722816 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.722822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.724656 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:52.724666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.726215 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.726223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:52.726399 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.726430 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 06:53:52.726524 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:53:52.726668 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":331,"threads_started":1}
I20260619 06:53:52.728631 362 leader_election.cc:302] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c; no voters:
I20260619 06:53:52.729038 497 raft_consensus.cc:2781] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:53:52.729192 497 raft_consensus.cc:482] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:53:52.729333 497 raft_consensus.cc:3037] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:52.729909 32628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b29e26923ee4d67be493ba907f1a6aa" candidate_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d18d485ca8417b8a587c341707f872" is_pre_election: true
I20260619 06:53:52.730401 32628 raft_consensus.cc:2454] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eca9a8e0bf5541539222fdb2b9651e7c in term 0.
I20260619 06:53:52.731971 497 raft_consensus.cc:504] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.732728 497 leader_election.cc:288] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [CANDIDATE]: Term 1 election: Requested vote from peers 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747), 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
I20260619 06:53:52.733140 32762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b29e26923ee4d67be493ba907f1a6aa" candidate_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d8d9be338e8498b845bad51c017ec31"
I20260619 06:53:52.733181 32628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b29e26923ee4d67be493ba907f1a6aa" candidate_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d18d485ca8417b8a587c341707f872"
I20260619 06:53:52.733410 32628 raft_consensus.cc:3037] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:52.733388 32762 raft_consensus.cc:3037] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:52.735582 32628 raft_consensus.cc:2454] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca9a8e0bf5541539222fdb2b9651e7c in term 1.
I20260619 06:53:52.735589 32762 raft_consensus.cc:2454] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eca9a8e0bf5541539222fdb2b9651e7c in term 1.
I20260619 06:53:52.735949 362 leader_election.cc:302] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16d18d485ca8417b8a587c341707f872, eca9a8e0bf5541539222fdb2b9651e7c; no voters:
I20260619 06:53:52.736229 497 raft_consensus.cc:2781] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:52.736658 497 raft_consensus.cc:686] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 1 LEADER]: Becoming Leader. State: Replica: eca9a8e0bf5541539222fdb2b9651e7c, State: Running, Role: LEADER
I20260619 06:53:52.737150 497 consensus_queue.cc:237] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } }
I20260619 06:53:52.742019 32483 catalog_manager.cc:5495] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c reported cstate change: term changed from 0 to 1, leader changed from <none> to eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67). New cstate: current_term: 1 leader_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } health_report { overall_health: UNKNOWN } } }
I20260619 06:53:52.764222 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.759143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35344 (local address 127.18.39.67:44775)
0619 06:53:52.759328 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.759332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.759352 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:53:52.759445 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.759449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.759525 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.759649 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:53:52.759986 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.760699 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.762117 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.762515 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.762607 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.763087 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:53:52.763617 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:53:52.763726 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.763917 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.764000 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:53:52.767941 32674 tablet.cc:2369] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:52.773161 32628 raft_consensus.cc:1261] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Refusing update from remote peer eca9a8e0bf5541539222fdb2b9651e7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:52.773427 32762 raft_consensus.cc:1261] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Refusing update from remote peer eca9a8e0bf5541539222fdb2b9651e7c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:52.774406 501 consensus_queue.cc:1035] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:52.774832 497 consensus_queue.cc:1035] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:52.799700 32483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:52.799971 32483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:52.803510 32483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Sending DeleteTablet for 3 replicas of tablet 3b29e26923ee4d67be493ba907f1a6aa
I20260619 06:53:52.804682 408 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:52 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:35332
I20260619 06:53:52.804795 32742 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:52 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38492
I20260619 06:53:52.804924 32608 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b29e26923ee4d67be493ba907f1a6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:52 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:33520
I20260619 06:53:52.806018 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32545
I20260619 06:53:52.815979 516 tablet_replica.cc:330] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: stopping tablet replica
I20260619 06:53:52.818019 516 raft_consensus.cc:2229] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:52.819154 516 raft_consensus.cc:2258] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:52.823494 516 ts_tablet_manager.cc:1905] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:52.826203 515 tablet_replica.cc:330] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: stopping tablet replica
I20260619 06:53:52.826258 514 tablet_replica.cc:330] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: stopping tablet replica
I20260619 06:53:52.826670 514 raft_consensus.cc:2229] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:53:52.826742 515 raft_consensus.cc:2229] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:52.827371 515 raft_consensus.cc:2258] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:52.827371 514 raft_consensus.cc:2258] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 19 06:53:52 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:53:52.829581 514 ts_tablet_manager.cc:1905] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:52.829578 515 ts_tablet_manager.cc:1905] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:52.836862 515 ts_tablet_manager.cc:1918] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:52.837106 515 log.cc:1196] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/3b29e26923ee4d67be493ba907f1a6aa
I20260619 06:53:52.837718 515 ts_tablet_manager.cc:1939] T 3b29e26923ee4d67be493ba907f1a6aa P 5d8d9be338e8498b845bad51c017ec31: Deleting consensus metadata
I20260619 06:53:52.837961 516 ts_tablet_manager.cc:1918] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:52.838168 516 log.cc:1196] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/3b29e26923ee4d67be493ba907f1a6aa
I20260619 06:53:52.838745 516 ts_tablet_manager.cc:1939] T 3b29e26923ee4d67be493ba907f1a6aa P 16d18d485ca8417b8a587c341707f872: Deleting consensus metadata
I20260619 06:53:52.840029 32469 catalog_manager.cc:4841] TS 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): tablet 3b29e26923ee4d67be493ba907f1a6aa (table test-table [id=06033216387941e892c338b932348636]) successfully deleted
I20260619 06:53:52.840433 32469 catalog_manager.cc:4841] TS 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631): tablet 3b29e26923ee4d67be493ba907f1a6aa (table test-table [id=06033216387941e892c338b932348636]) successfully deleted
I20260619 06:53:52.842233 490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.807421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.18.39.65:42631)
0619 06:53:52.807654 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.807662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.816357 (+ 8695us) server_negotiation.cc:407] Connection header received
0619 06:53:52.816665 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.816671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.816773 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.816871 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:52.819076 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.820324 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.822522 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.823206 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.823309 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.834123 (+ 10814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:52.834142 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:52.834162 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:52.834217 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:52.838544 (+ 4327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.839598 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.839610 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.839613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.839798 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.840277 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.840282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.840284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.840848 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:52.841182 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.841531 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.841748 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:53:52.842625 514 ts_tablet_manager.cc:1918] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:52.842805 514 log.cc:1196] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/3b29e26923ee4d67be493ba907f1a6aa
I20260619 06:53:52.842873 32668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:52.843389 514 ts_tablet_manager.cc:1939] T 3b29e26923ee4d67be493ba907f1a6aa P eca9a8e0bf5541539222fdb2b9651e7c: Deleting consensus metadata
I20260619 06:53:52.845081 32469 catalog_manager.cc:4841] TS eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): tablet 3b29e26923ee4d67be493ba907f1a6aa (table test-table [id=06033216387941e892c338b932348636]) successfully deleted
I20260619 06:53:52.935482 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32545
I20260619 06:53:52.953692 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32679
Jun 19 06:53:52 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:53:52.969722 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:52.954553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.18.39.66:41747)
0619 06:53:52.954730 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:52.954738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:52.954761 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:53:52.955001 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:52.955006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:52.955086 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:52.955194 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:52.956473 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.957186 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.958633 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:52.959025 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:52.959162 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:52.963882 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:52.963896 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:52.963920 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:52.963956 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:52.966824 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.967554 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.967562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.967564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.967693 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:52.968073 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:52.968083 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:52.968085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:52.968456 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:52.968754 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:52.969066 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 06:53:52.969286 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:53:52.970487 334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:53.064431 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32679
I20260619 06:53:53.088733 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 345
Jun 19 06:53:53 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:53:53.107316 486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:53.089574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.18.39.67:44775)
0619 06:53:53.089760 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:53.089771 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:53.089815 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:53:53.090052 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:53.090060 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:53.090152 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:53.090259 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:53.091554 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:53.092449 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:53.093855 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:53.094309 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:53.094484 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:53.099312 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:53.099332 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:53.099363 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:53.099418 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:53.103841 (+ 4423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:53.104692 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:53.104703 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:53.104707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:53.104972 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:53.105528 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:53.105534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:53.105538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:53.106020 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:53.106324 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:53.106610 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 06:53:53.106858 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:53:53.108111 468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:53.233546 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 345
I20260619 06:53:53.252676 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 32452
I20260619 06:53:53.253952 32514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:53.369192 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 32452
I20260619 06:53:53.390803 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--webserver_interface=127.18.39.126
--webserver_port=34521
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:53.733210 522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:53.733542 522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:53.733712 522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:53.741169 522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:53.741310 522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:53.741431 522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:53.741545 522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:53.741632 522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:53.752007 522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=34521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:53.753451 522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:53.754829 522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:53.778242 529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:53.778406 531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:53.778442 528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:53.779297 522 server_base.cc:1034] running on GCE node
I20260619 06:53:53.780236 522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:53.782029 522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:53.783255 522 hybrid_clock.cc:648] HybridClock initialized: now 1781852033783177 us; error 65 us; skew 500 ppm
I20260619 06:53:53.783727 522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:53 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852033, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:53.790627 522 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:53.792503 522 webserver.cc:460] Webserver started at http://127.18.39.126:34521/ using document root <none> and password file <none>
I20260619 06:53:53.793280 522 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:53.793493 522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:53.799880 522 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 06:53:53.803371 538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:53.804951 522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:53:53.805159 522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "77ba538586884539b1fb48fb67a35ee8"
format_stamp: "Formatted at 2026-06-19 06:53:50 on dist-test-slave-1h5s"
server_key: "2618a22064c85300e1e6944f355e723a"
server_key_iv: "1b593c787b700c2bea4e268025908e64"
server_key_version: "encryptionkey@0"
I20260619 06:53:53.806413 522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:53.829696 522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:53.834444 522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:53.834867 522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:53.854763 522 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:41087
I20260619 06:53:53.854852 589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:41087 every 8 connection(s)
I20260619 06:53:53.856370 522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:53.864228 590 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:53:53.865463 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 522
I20260619 06:53:53.866549 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:42631
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=35765
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:53.870234 590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap starting.
I20260619 06:53:53.894142 590 log.cc:824] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:53.907670 590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:53:53.908299 590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap complete.
I20260619 06:53:53.918893 590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:53.919754 590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Initialized, Role: FOLLOWER
I20260619 06:53:53.920532 590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:53.920780 590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:53.920915 590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:53.921113 590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:53:53.925127 590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:53.925715 590 leader_election.cc:302] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77ba538586884539b1fb48fb67a35ee8; no voters:
I20260619 06:53:53.926383 590 leader_election.cc:288] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:53:53.926530 594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:53:53.927783 594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 LEADER]: Becoming Leader. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Running, Role: LEADER
I20260619 06:53:53.928371 594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:53.928911 590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:53.931023 596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77ba538586884539b1fb48fb67a35ee8. Latest consensus state: current_term: 2 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:53.931428 596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:53.931313 595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:53.931738 595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:53.943049 601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:53.951726 601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:53.953290 601 catalog_manager.cc:1259] Loaded cluster ID: 94dd0ba78a664d8991e222d2ddee9ede
I20260619 06:53:53.953419 601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:53:53.957104 601 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 06:53:53.974951 601 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 06:53:53.997397 601 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:53:53.999275 601 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Loaded TSK: 0
I20260619 06:53:54.001061 601 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:53:54.237499 592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:54.237833 592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:54.237998 592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:54.245555 592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:54.245711 592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:54.245887 592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:53:54.256074 592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:42631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=35765
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:54.257572 592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:54.258934 592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:54.274183 618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:54.274307 617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:54.275009 620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:54.275290 592 server_base.cc:1034] running on GCE node
I20260619 06:53:54.276005 592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:54.277020 592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:54.278251 592 hybrid_clock.cc:648] HybridClock initialized: now 1781852034278180 us; error 68 us; skew 500 ppm
I20260619 06:53:54.278584 592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:54 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852034, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:54.281858 592 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:53:54.283581 592 webserver.cc:460] Webserver started at http://127.18.39.65:35765/ using document root <none> and password file <none>
I20260619 06:53:54.284297 592 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:54.284446 592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:54.290071 592 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:53:54.292933 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.294462 592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:53:54.294648 592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "16d18d485ca8417b8a587c341707f872"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "c0a09219bf4442ba6df0284eea66e8e1"
server_key_iv: "6b7a8af2af85ad2e695468571932f363"
server_key_version: "encryptionkey@0"
I20260619 06:53:54.295637 592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:54.331692 592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:54.337518 592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:54.337821 592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:54.339031 592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:54.340456 592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:54.340559 592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.340723 592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:54.340821 592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.380748 592 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42631
I20260619 06:53:54.380918 739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42631 every 8 connection(s)
I20260619 06:53:54.382349 592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:53:54.384091 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 592
I20260619 06:53:54.384891 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:41747
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=39933
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:54 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852034, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:54.416074 743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:54.385688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:55225)
0619 06:53:54.390279 (+ 4591us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:54.390355 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:54.393983 (+ 3628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:54.395604 (+ 1621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:54.395616 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:54.396347 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:54.396928 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:54.396948 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.398846 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.398855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:54.401131 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:54.401151 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.402359 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.402368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:54.402705 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:54.403588 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:54.403632 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:54.406630 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:54.411838 (+ 5208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:54.411845 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:54.411858 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:54.412326 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:54.412862 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:54.412865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:54.412867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:54.413070 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:54.414021 (+ 951us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:54.414028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:54.414340 (+ 312us) client_negotiation.cc:769] Sending connection context
0619 06:53:54.414582 (+ 242us) client_negotiation.cc:240] Negotiation successful
0619 06:53:54.414983 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4107,"thread_start_us":3820,"threads_started":1}
I20260619 06:53:54.416340 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:54.385458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:55225 (local address 127.18.39.126:41087)
0619 06:53:54.393943 (+ 8485us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:54.393963 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:54.394026 (+ 63us) server_negotiation.cc:407] Connection header received
0619 06:53:54.394616 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:54.394631 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:54.395025 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:54.395434 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:54.397192 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.398621 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.401321 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.402093 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.402244 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:54.406971 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:54.406985 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:54.407010 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:54.407070 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:54.411644 (+ 4574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:54.412486 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:54.412496 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:54.412503 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:54.412736 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:54.413191 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:54.413196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:54.413199 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:54.413872 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:54.414049 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:54.414802 (+ 753us) server_negotiation.cc:299] Negotiation successful
0619 06:53:54.415225 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8199,"thread_start_us":8055,"threads_started":1}
I20260619 06:53:54.419704 740 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
I20260619 06:53:54.420028 740 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:54.420776 740 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
I20260619 06:53:54.423223 555 ts_manager.cc:194] Registered new tserver with Master: 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
I20260619 06:53:54.425208 555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:55225
W20260619 06:53:54.721829 744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:54.722121 744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:54.722218 744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:54.729534 744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:54.729663 744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:54.729840 744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:53:54.739635 744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:41747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=39933
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:54.740995 744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:54.742308 744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:54.755906 750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:54.755983 752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:54.755864 749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:54.756614 744 server_base.cc:1034] running on GCE node
I20260619 06:53:54.757221 744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:54.758230 744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:54.759464 744 hybrid_clock.cc:648] HybridClock initialized: now 1781852034759418 us; error 29 us; skew 500 ppm
I20260619 06:53:54.759819 744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:54 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852034, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:54.763161 744 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:53:54.764629 744 webserver.cc:460] Webserver started at http://127.18.39.66:39933/ using document root <none> and password file <none>
I20260619 06:53:54.765389 744 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:54.765556 744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:54.770982 744 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:53:54.774215 759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.775763 744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:53:54.775974 744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "5d8d9be338e8498b845bad51c017ec31"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "1f03b2ccf9a7829737d53fd070363219"
server_key_iv: "9a6db13588fcfd5d77f4f987862c9f8d"
server_key_version: "encryptionkey@0"
I20260619 06:53:54.777057 744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:54.794972 744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:54.799166 744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:54.799453 744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:54.800498 744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:54.802021 744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:54.802145 744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.802285 744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:54.802376 744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:54.835634 744 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:41747
I20260619 06:53:54.835691 871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:41747 every 8 connection(s)
I20260619 06:53:54.837162 744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:53:54.842094 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 744
I20260619 06:53:54.842751 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:44775
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=43083
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:53:54 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852034, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:54.859200 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:54.839679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47117 (local address 127.18.39.126:41087)
0619 06:53:54.839862 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:54.839866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:54.841018 (+ 1152us) server_negotiation.cc:407] Connection header received
0619 06:53:54.842895 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:54.842900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:54.843012 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:54.843132 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:54.844683 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.845486 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.847028 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.847460 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.847594 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:54.851453 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:54.851462 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:54.851470 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:54.851520 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:54.855281 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:54.856250 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:54.856257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:54.856263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:54.856423 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:54.856916 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:54.856924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:54.856930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:54.857380 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:54.857538 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:54.858413 (+ 875us) server_negotiation.cc:299] Negotiation successful
0619 06:53:54.858724 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:53:54.860221 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:54.840157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:47117)
0619 06:53:54.840816 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:54.840855 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:54.842628 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:54.843375 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:54.843384 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:54.843869 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:54.844511 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:54.844528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.845633 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.845639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:54.846864 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:54.846875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:54.847692 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:54.847699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:54.847940 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:54.848667 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:54.848689 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:54.851178 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:54.855465 (+ 4287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:54.855475 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:54.855490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:54.856106 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:54.856548 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:54.856553 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:54.856558 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:54.856782 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:54.857522 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:54.857531 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:54.857962 (+ 431us) client_negotiation.cc:769] Sending connection context
0619 06:53:54.858269 (+ 307us) client_negotiation.cc:240] Negotiation successful
0619 06:53:54.858847 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":264,"threads_started":1}
I20260619 06:53:54.861964 872 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
I20260619 06:53:54.862288 872 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:54.863138 872 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
I20260619 06:53:54.864909 555 ts_manager.cc:194] Registered new tserver with Master: 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
I20260619 06:53:54.866297 555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:47117
W20260619 06:53:55.187680 875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:55.187980 875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:55.188149 875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:55.195789 875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:55.195922 875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:55.196101 875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:53:55.206228 875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:44775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=43083
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:55.207597 875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:55.209034 875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:55.222759 880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:55.222810 881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:55.222841 883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:55.224207 875 server_base.cc:1034] running on GCE node
I20260619 06:53:55.224831 875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:55.225878 875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:55.227092 875 hybrid_clock.cc:648] HybridClock initialized: now 1781852035227035 us; error 44 us; skew 500 ppm
I20260619 06:53:55.227383 875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:55 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852035, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:55.234295 875 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:53:55.235903 875 webserver.cc:460] Webserver started at http://127.18.39.67:43083/ using document root <none> and password file <none>
I20260619 06:53:55.236632 875 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:55.236790 875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:55.242445 875 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:53:55.245325 890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:55.246909 875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:53:55.247098 875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
format_stamp: "Formatted at 2026-06-19 06:53:52 on dist-test-slave-1h5s"
server_key: "b10b08fb0df403aa447a2d789c7d87cc"
server_key_iv: "6f4206d2bfed8aaff2e96c641d1844b4"
server_key_version: "encryptionkey@0"
I20260619 06:53:55.248081 875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:55.265503 875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:55.270088 875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:55.270381 875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:55.271426 875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:55.272804 875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:55.272898 875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:55.273018 875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:55.273099 875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:55.307049 875 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44775
I20260619 06:53:55.307139 1002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44775 every 8 connection(s)
I20260619 06:53:55.308601 875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:53:55.309962 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 875
Jun 19 06:53:55 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852035, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:53:55.326700 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.311157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.18.39.126:41087)
0619 06:53:55.311398 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.311407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.311832 (+ 425us) server_negotiation.cc:407] Connection header received
0619 06:53:55.312055 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.312061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.312139 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.312248 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:55.313973 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.314771 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.316387 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.316820 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.316959 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.320528 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:55.320534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:55.320537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:55.320576 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:55.323648 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.324558 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.324565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.324569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.324730 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.325147 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.325153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.325156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.325535 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:55.325848 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.326091 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.326342 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:53:55.333065 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.312075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:34111 (local address 127.18.39.126:41087)
0619 06:53:55.312665 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.312671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.313479 (+ 808us) server_negotiation.cc:407] Connection header received
0619 06:53:55.315440 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.315447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.315531 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.315643 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:55.317251 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.318389 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.320946 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.321536 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.321694 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.325655 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:55.325662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:55.325666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:55.325705 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:55.329578 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.330482 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.330489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.330493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.330631 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.331117 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.331123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.331127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.331427 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:55.331593 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.332390 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.332633 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":386,"threads_started":1}
I20260619 06:53:55.334329 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.312409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:34111)
0619 06:53:55.313276 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.313354 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.315108 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.315930 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.315940 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.316500 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:55.317086 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.317104 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.318619 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.318628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.320773 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.320787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.321795 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.321816 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.322112 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.322886 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:55.322910 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:55.325398 (+ 2488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:55.329750 (+ 4352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:55.329760 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:55.329782 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:55.330353 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:55.330766 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:55.330771 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:55.330775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:55.330995 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:55.331580 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:55.331588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:55.332002 (+ 414us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.332312 (+ 310us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.332915 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":352,"threads_started":1}
I20260619 06:53:55.335908 1003 heartbeater.cc:347] Connected to a master server at 127.18.39.126:41087
I20260619 06:53:55.336256 1003 heartbeater.cc:464] Registering TS with master...
I20260619 06:53:55.337280 1003 heartbeater.cc:511] Master 127.18.39.126:41087 requested a full tablet report, sending...
I20260619 06:53:55.339154 555 ts_manager.cc:194] Registered new tserver with Master: eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775)
I20260619 06:53:55.340370 555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:34111
I20260619 06:53:55.342567 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:53:55.349237 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.343516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.18.39.126:41087)
0619 06:53:55.343702 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.343706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.344164 (+ 458us) server_negotiation.cc:407] Connection header received
0619 06:53:55.344323 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.344327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.344463 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.344580 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:53:55.344989 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.345732 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.347125 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.347554 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.347652 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.348171 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:53:55.348693 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:53:55.348804 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.348965 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.349036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:53:55.350227 555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:53:55.358026 555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:53:55.383601 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.372218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33540)
0619 06:53:55.372776 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.372799 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.372950 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.374104 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.374108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.374131 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:55.374572 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.374584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.376504 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.376510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.379129 (+ 2619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.379141 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.381034 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.381040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.381230 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.381381 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.381514 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.381751 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":105,"thread_start_us":288,"threads_started":1}
I20260619 06:53:55.384447 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.370783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44775 (local address 127.0.0.1:35356)
0619 06:53:55.371854 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.371896 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.372068 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.373116 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.373126 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.373160 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:55.373613 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.374344 (+ 731us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.376504 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.376510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.379823 (+ 3313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.379837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.383372 (+ 3535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.383382 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.383573 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.383624 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.383841 (+ 217us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.384008 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":88,"thread_start_us":546,"threads_started":1}
I20260619 06:53:55.384991 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.373283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.18.39.66:41747)
0619 06:53:55.375771 (+ 2488us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.375778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.375801 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:53:55.375874 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.375879 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.376057 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.376254 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:55.377021 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.378527 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.381200 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.383045 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.383235 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.383489 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.384511 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.384706 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2343,"thread_start_us":1955,"threads_started":1}
I20260619 06:53:55.385280 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.371087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.18.39.67:44775)
0619 06:53:55.372151 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.372160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.372195 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:53:55.372292 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.372301 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.372590 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.372776 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:55.374571 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.376334 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.380030 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.381511 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.381631 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.381874 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.384665 (+ 2791us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.384828 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":679,"threads_started":1}
I20260619 06:53:55.385321 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.372826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:41747 (local address 127.0.0.1:38504)
0619 06:53:55.373816 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.373832 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.373939 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.376241 (+ 2302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.376245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.376270 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:55.376803 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.376821 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.378696 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.378702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.380945 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.380964 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.383658 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.383666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.383868 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.383897 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.383969 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.384077 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":4,"thread_start_us":752,"threads_started":1}
I20260619 06:53:55.389496 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.372553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.18.39.65:42631)
0619 06:53:55.373502 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.373513 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.373540 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:53:55.373651 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.373660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.373900 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.374127 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:55.374772 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.376334 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.379292 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.380780 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.384287 (+ 3507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.384510 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.384715 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.384886 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":626,"threads_started":1}
I20260619 06:53:55.389276 938 tablet_service.cc:1467] Processing CreateTablet for tablet 835b50ef051442929c892f2cac670a58 (DEFAULT_TABLE table=test-table [id=e17a3bf6559c497499d5102c495cf0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:55.389277 807 tablet_service.cc:1467] Processing CreateTablet for tablet 835b50ef051442929c892f2cac670a58 (DEFAULT_TABLE table=test-table [id=e17a3bf6559c497499d5102c495cf0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:55.390818 807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835b50ef051442929c892f2cac670a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:55.390818 938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835b50ef051442929c892f2cac670a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:55.392647 675 tablet_service.cc:1467] Processing CreateTablet for tablet 835b50ef051442929c892f2cac670a58 (DEFAULT_TABLE table=test-table [id=e17a3bf6559c497499d5102c495cf0e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:53:55.394582 675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835b50ef051442929c892f2cac670a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:53:55.406960 1019 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Bootstrap starting.
I20260619 06:53:55.409512 1020 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Bootstrap starting.
I20260619 06:53:55.411082 1021 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Bootstrap starting.
I20260619 06:53:55.412688 1019 tablet_bootstrap.cc:654] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:55.413740 1020 tablet_bootstrap.cc:654] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:55.414086 1019 log.cc:824] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:55.414644 1021 tablet_bootstrap.cc:654] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Neither blocks nor log segments found. Creating new log.
I20260619 06:53:55.414795 1020 log.cc:824] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:55.416015 1021 log.cc:824] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:55.417564 1019 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: No bootstrap required, opened a new log
I20260619 06:53:55.417834 1019 ts_tablet_manager.cc:1397] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260619 06:53:55.418207 1020 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: No bootstrap required, opened a new log
I20260619 06:53:55.418545 1020 ts_tablet_manager.cc:1397] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:53:55.425572 1019 raft_consensus.cc:348] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.426100 1019 raft_consensus.cc:374] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:55.426220 1019 raft_consensus.cc:729] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d18d485ca8417b8a587c341707f872, State: Initialized, Role: FOLLOWER
I20260619 06:53:55.426903 1019 consensus_queue.cc:260] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.428089 740 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:55.428653 1019 ts_tablet_manager.cc:1428] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 06:53:55.429525 1021 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: No bootstrap required, opened a new log
I20260619 06:53:55.429859 1021 ts_tablet_manager.cc:1397] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.008s
I20260619 06:53:55.430752 1020 raft_consensus.cc:348] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.431141 1020 raft_consensus.cc:374] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:55.431269 1020 raft_consensus.cc:729] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca9a8e0bf5541539222fdb2b9651e7c, State: Initialized, Role: FOLLOWER
I20260619 06:53:55.431915 1020 consensus_queue.cc:260] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.434152 1020 ts_tablet_manager.cc:1428] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260619 06:53:55.435122 1003 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:55.439189 1021 raft_consensus.cc:348] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.439630 1021 raft_consensus.cc:374] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:53:55.439738 1021 raft_consensus.cc:729] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8d9be338e8498b845bad51c017ec31, State: Initialized, Role: FOLLOWER
I20260619 06:53:55.440474 1021 consensus_queue.cc:260] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.441619 872 heartbeater.cc:503] Master 127.18.39.126:41087 was elected leader, sending a full tablet report...
I20260619 06:53:55.442384 1021 ts_tablet_manager.cc:1428] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260619 06:53:55.560644 1004 tablet.cc:2369] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:53:55.589210 873 tablet.cc:2369] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:55.626374 1027 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:55.626744 1027 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.628388 1027 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
W20260619 06:53:55.634883 741 tablet.cc:2369] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:53:55.636685 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.630167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:39841 (local address 127.18.39.65:42631)
0619 06:53:55.630371 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.630375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.630452 (+ 77us) server_negotiation.cc:407] Connection header received
0619 06:53:55.630662 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.630665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.630721 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.630793 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:55.631317 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.632264 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.634723 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.635692 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.635905 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.635984 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.636303 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.636434 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 06:53:55.636847 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.628922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:54463 (local address 127.18.39.67:44775)
0619 06:53:55.629169 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.629178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.629672 (+ 494us) server_negotiation.cc:407] Connection header received
0619 06:53:55.629898 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.629905 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.629990 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.630082 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:53:55.630901 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.631956 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.634618 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.635842 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.636028 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.636110 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.636459 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.636597 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:53:55.637267 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.629762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.18.39.66:39841)
0619 06:53:55.630333 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.630353 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.630530 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.630841 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.630845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.630872 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:55.631156 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.631163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.632389 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.632402 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.634504 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.634517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.635910 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.635916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.636114 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.636139 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.636256 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.636390 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":6,"spinlock_wait_cycles":1408,"thread_start_us":285,"threads_started":1}
I20260619 06:53:55.637286 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.628827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44775 (local address 127.18.39.66:54463)
0619 06:53:55.629559 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.629575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:55.629754 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:55.630273 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:55.630278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:55.630307 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:53:55.630738 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.630748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.632078 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.632082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.634424 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:55.634445 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.636092 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.636099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:55.636266 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.636320 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 06:53:55.636432 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 06:53:55.636570 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":25,"spinlock_wait_cycles":44672,"thread_start_us":406,"threads_started":1}
I20260619 06:53:55.638317 958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835b50ef051442929c892f2cac670a58" candidate_uuid: "5d8d9be338e8498b845bad51c017ec31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" is_pre_election: true
I20260619 06:53:55.638315 695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835b50ef051442929c892f2cac670a58" candidate_uuid: "5d8d9be338e8498b845bad51c017ec31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d18d485ca8417b8a587c341707f872" is_pre_election: true
I20260619 06:53:55.638829 958 raft_consensus.cc:2454] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d8d9be338e8498b845bad51c017ec31 in term 0.
I20260619 06:53:55.638828 695 raft_consensus.cc:2454] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d8d9be338e8498b845bad51c017ec31 in term 0.
I20260619 06:53:55.639508 761 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c; no voters:
I20260619 06:53:55.639894 1027 raft_consensus.cc:2781] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:53:55.640043 1027 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:53:55.640154 1027 raft_consensus.cc:3037] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:55.642424 1027 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.643097 1027 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [CANDIDATE]: Term 1 election: Requested vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631)
I20260619 06:53:55.643579 958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835b50ef051442929c892f2cac670a58" candidate_uuid: "5d8d9be338e8498b845bad51c017ec31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
I20260619 06:53:55.643695 695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835b50ef051442929c892f2cac670a58" candidate_uuid: "5d8d9be338e8498b845bad51c017ec31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16d18d485ca8417b8a587c341707f872"
I20260619 06:53:55.643785 958 raft_consensus.cc:3037] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:55.643934 695 raft_consensus.cc:3037] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:53:55.646129 958 raft_consensus.cc:2454] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d8d9be338e8498b845bad51c017ec31 in term 1.
I20260619 06:53:55.646286 695 raft_consensus.cc:2454] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d8d9be338e8498b845bad51c017ec31 in term 1.
I20260619 06:53:55.646569 761 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c; no voters:
I20260619 06:53:55.646849 1027 raft_consensus.cc:2781] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:53:55.647279 1027 raft_consensus.cc:686] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 1 LEADER]: Becoming Leader. State: Replica: 5d8d9be338e8498b845bad51c017ec31, State: Running, Role: LEADER
I20260619 06:53:55.647763 1027 consensus_queue.cc:237] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:55.652516 555 catalog_manager.cc:5495] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66). New cstate: current_term: 1 leader_uuid: "5d8d9be338e8498b845bad51c017ec31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } health_report { overall_health: HEALTHY } } }
I20260619 06:53:55.679663 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.674035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.18.39.66:41747)
0619 06:53:55.674245 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.674251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.674270 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:53:55.674534 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.674538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.674625 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.674774 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:53:55.675177 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.675891 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.677470 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.677879 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.678023 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.678541 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:53:55.679059 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:53:55.679218 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.679277 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.679360 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:53:55.688038 958 raft_consensus.cc:1261] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Refusing update from remote peer 5d8d9be338e8498b845bad51c017ec31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:55.688040 695 raft_consensus.cc:1261] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Refusing update from remote peer 5d8d9be338e8498b845bad51c017ec31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:53:55.688990 1030 consensus_queue.cc:1035] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:55.689391 1027 consensus_queue.cc:1035] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:53:55.714723 555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:55.715008 555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:53:55.718678 555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Sending DeleteTablet for 3 replicas of tablet 835b50ef051442929c892f2cac670a58
I20260619 06:53:55.720026 675 tablet_service.cc:1514] Processing DeleteTablet for tablet 835b50ef051442929c892f2cac670a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:55 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:33540
I20260619 06:53:55.720166 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 592
I20260619 06:53:55.720322 807 tablet_service.cc:1514] Processing DeleteTablet for tablet 835b50ef051442929c892f2cac670a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:55 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38504
I20260619 06:53:55.720970 938 tablet_service.cc:1514] Processing DeleteTablet for tablet 835b50ef051442929c892f2cac670a58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:53:55 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:35356
I20260619 06:53:55.731865 1045 tablet_replica.cc:330] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: stopping tablet replica
I20260619 06:53:55.732403 1045 raft_consensus.cc:2229] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:55.732980 1045 raft_consensus.cc:2258] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:55.734033 1043 tablet_replica.cc:330] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: stopping tablet replica
I20260619 06:53:55.734548 1043 raft_consensus.cc:2229] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:53:55.735100 1043 raft_consensus.cc:2258] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:55.736948 1045 ts_tablet_manager.cc:1905] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:55.737787 1044 tablet_replica.cc:330] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: stopping tablet replica
I20260619 06:53:55.738338 1044 raft_consensus.cc:2229] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:53:55.739099 1044 raft_consensus.cc:2258] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:53:55.741427 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.721879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.18.39.65:42631)
0619 06:53:55.722097 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.722107 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.722132 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:53:55.722207 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.722216 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.722323 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.722445 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:55.724300 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.725883 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.727836 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.728744 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.728881 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.732874 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:55.732908 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:55.732928 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:55.732981 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:55.737071 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.738082 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.738091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.738094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.738253 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.738682 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.738688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.738691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.739190 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:55.739478 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.740719 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.740956 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 06:53:55.742636 735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:55.743748 1045 ts_tablet_manager.cc:1918] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:55.743928 1045 log.cc:1196] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/835b50ef051442929c892f2cac670a58
I20260619 06:53:55.744519 1045 ts_tablet_manager.cc:1939] T 835b50ef051442929c892f2cac670a58 P eca9a8e0bf5541539222fdb2b9651e7c: Deleting consensus metadata
I20260619 06:53:55.746392 540 catalog_manager.cc:4841] TS eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): tablet 835b50ef051442929c892f2cac670a58 (table test-table [id=e17a3bf6559c497499d5102c495cf0e9]) successfully deleted
I20260619 06:53:55.741446 1044 ts_tablet_manager.cc:1905] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:53:55.754107 1044 ts_tablet_manager.cc:1918] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:53:55.754312 1044 log.cc:1196] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/835b50ef051442929c892f2cac670a58
I20260619 06:53:55.754917 1044 ts_tablet_manager.cc:1939] T 835b50ef051442929c892f2cac670a58 P 5d8d9be338e8498b845bad51c017ec31: Deleting consensus metadata
I20260619 06:53:55.756891 540 catalog_manager.cc:4841] TS 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): tablet 835b50ef051442929c892f2cac670a58 (table test-table [id=e17a3bf6559c497499d5102c495cf0e9]) successfully deleted
I20260619 06:53:55.846282 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 592
W20260619 06:53:55.868031 540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.39.65:42631 (error 108)
I20260619 06:53:55.872370 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.871931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33556)
0619 06:53:55.872091 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.872216 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 06:53:55.872473 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 744
W20260619 06:53:55.872773 540 catalog_manager.cc:4584] TS 16d18d485ca8417b8a587c341707f872 (127.18.39.65:42631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 835b50ef051442929c892f2cac670a58: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
I20260619 06:53:55.890666 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.873248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38522 (local address 127.18.39.66:41747)
0619 06:53:55.873585 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:55.873591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:55.873613 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:53:55.873686 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:55.873692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:55.873769 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:55.873890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:55.875680 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.876972 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.878968 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:55.879656 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:55.879792 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:55.883508 (+ 3716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:55.883528 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:55.883550 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:55.883602 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:55.887762 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.888479 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.888494 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.888498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.888667 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:55.889024 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:55.889030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:55.889034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:55.889521 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:55.889852 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:55.890077 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 06:53:55.890286 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260619 06:53:55.891412 867 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:55.927868 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.927249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33572)
0619 06:53:55.927531 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.927664 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260619 06:53:55.929087 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.928657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33586)
0619 06:53:55.928830 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.928944 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 06:53:55.989691 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.989078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33592)
0619 06:53:55.989361 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.989474 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 06:53:55.990794 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:55.990410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33604)
0619 06:53:55.990580 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:55.990665 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 06:53:56.007256 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 744
I20260619 06:53:56.027190 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 875
I20260619 06:53:56.040853 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:56.027986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35368 (local address 127.18.39.67:44775)
0619 06:53:56.028176 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:56.028182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:56.028280 (+ 98us) server_negotiation.cc:407] Connection header received
0619 06:53:56.028514 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:56.028518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:56.028581 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:56.028689 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:56.030172 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:56.030915 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:56.032280 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:56.032677 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:56.032809 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:56.035411 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:56.035432 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:56.035447 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:56.035479 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:56.038144 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:56.038838 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:56.038846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:56.038848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:56.038983 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:56.039353 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:56.039362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:56.039366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:56.039691 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:56.040003 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:56.040295 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 06:53:56.040504 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:53:56.041743 998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:56.071676 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:56.071057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33612)
0619 06:53:56.071305 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:56.071444 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 06:53:56.072904 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:56.072439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33618)
0619 06:53:56.072679 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:56.072769 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260619 06:53:56.132266 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 875
I20260619 06:53:56.148910 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 522
I20260619 06:53:56.150123 585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:53:56.264425 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:56.263659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42631 (local address 127.0.0.1:33620)
0619 06:53:56.263907 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:56.264030 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:53:56.264605 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 522
I20260619 06:53:56.286094 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--webserver_interface=127.18.39.126
--webserver_port=34521
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:53:56.623795 1051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:56.624099 1051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:56.624190 1051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:56.631487 1051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:53:56.631603 1051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:56.631649 1051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:56.631707 1051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:53:56.631750 1051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:53:56.641636 1051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=34521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:56.642920 1051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:56.644174 1051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:56.655903 1056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:56.655824 1057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:56.655897 1059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:56.656531 1051 server_base.cc:1034] running on GCE node
I20260619 06:53:56.657490 1051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:56.659170 1051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:56.660373 1051 hybrid_clock.cc:648] HybridClock initialized: now 1781852036660339 us; error 33 us; skew 500 ppm
I20260619 06:53:56.660754 1051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:56 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852036, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:56.664005 1051 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:53:56.665467 1051 webserver.cc:460] Webserver started at http://127.18.39.126:34521/ using document root <none> and password file <none>
I20260619 06:53:56.666119 1051 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:56.666216 1051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:56.671487 1051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 06:53:56.674278 1066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:56.675683 1051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:53:56.675855 1051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "77ba538586884539b1fb48fb67a35ee8"
format_stamp: "Formatted at 2026-06-19 06:53:50 on dist-test-slave-1h5s"
server_key: "2618a22064c85300e1e6944f355e723a"
server_key_iv: "1b593c787b700c2bea4e268025908e64"
server_key_version: "encryptionkey@0"
I20260619 06:53:56.676815 1051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:56.702749 1051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:56.707170 1051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:56.707461 1051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:56.726729 1051 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:41087
I20260619 06:53:56.726749 1117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:41087 every 8 connection(s)
I20260619 06:53:56.728246 1051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:53:56.729192 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1051
I20260619 06:53:56.730037 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:42631
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=35765
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:56.735531 1118 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:53:56.738919 1118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap starting.
I20260619 06:53:56.760651 1118 log.cc:824] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:56.776921 1118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:53:56.777606 1118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8: Bootstrap complete.
I20260619 06:53:56.787539 1118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:56.788491 1118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Initialized, Role: FOLLOWER
I20260619 06:53:56.789165 1118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:56.789482 1118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:53:56.789626 1118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:53:56.789870 1118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 2 FOLLOWER]: Advancing to term 3
I20260619 06:53:56.793870 1118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:56.794377 1118 leader_election.cc:302] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77ba538586884539b1fb48fb67a35ee8; no voters:
I20260619 06:53:56.795073 1118 leader_election.cc:288] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 06:53:56.795219 1122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 06:53:56.796552 1122 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [term 3 LEADER]: Becoming Leader. State: Replica: 77ba538586884539b1fb48fb67a35ee8, State: Running, Role: LEADER
I20260619 06:53:56.797176 1122 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } }
I20260619 06:53:56.797578 1118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:53:56.800076 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77ba538586884539b1fb48fb67a35ee8. Latest consensus state: current_term: 3 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:56.800150 1123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "77ba538586884539b1fb48fb67a35ee8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77ba538586884539b1fb48fb67a35ee8" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 41087 } } }
I20260619 06:53:56.800478 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:56.800572 1123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ba538586884539b1fb48fb67a35ee8 [sys.catalog]: This master's current role is: LEADER
I20260619 06:53:56.811113 1129 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:53:56.821105 1129 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:53:56.823045 1129 catalog_manager.cc:1259] Loaded cluster ID: 94dd0ba78a664d8991e222d2ddee9ede
I20260619 06:53:56.823256 1129 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 06:53:56.825641 1129 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f98dea489ad google::LogMessage::Fail() at ??:0
@ 0x7f98dea4b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f98dea484de google::LogMessage::Flush() at ??:0
@ 0x7f98dea49a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f98f447b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f98f4478976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f98f4500610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f98f4500439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f98e1694345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f98e169791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f98e1697799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f98e1672eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f98e494c6db start_thread at ??:0
@ 0x7f98dd8e261f clone at ??:0
W20260619 06:53:57.142944 1120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:57.143230 1120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:57.143325 1120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:57.150633 1120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:57.150756 1120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:57.150882 1120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:53:57.160924 1120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:42631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=35765
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:57.162387 1120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:57.163697 1120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:57.177755 1159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:57.177659 1162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:57.177726 1160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:57.178393 1120 server_base.cc:1034] running on GCE node
I20260619 06:53:57.179052 1120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:57.180063 1120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:57.181291 1120 hybrid_clock.cc:648] HybridClock initialized: now 1781852037181238 us; error 48 us; skew 500 ppm
I20260619 06:53:57.181648 1120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:57 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852037, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:57.184918 1120 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:53:57.186540 1120 webserver.cc:460] Webserver started at http://127.18.39.65:35765/ using document root <none> and password file <none>
I20260619 06:53:57.187225 1120 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:57.187362 1120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:57.192736 1120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260619 06:53:57.197184 1169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:57.198616 1120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 06:53:57.198807 1120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "16d18d485ca8417b8a587c341707f872"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "c0a09219bf4442ba6df0284eea66e8e1"
server_key_iv: "6b7a8af2af85ad2e695468571932f363"
server_key_version: "encryptionkey@0"
I20260619 06:53:57.199781 1120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:57.230432 1120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:57.234582 1120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:57.234879 1120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:57.235942 1120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:57.237892 1176 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260619 06:53:57.243757 1120 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 06:53:57.243938 1120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 06:53:57.244089 1120 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 06:53:57.247407 1120 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 06:53:57.247536 1120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260619 06:53:57.247972 1176 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Bootstrap starting.
I20260619 06:53:57.257302 1176 log.cc:824] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Log is configured to *not* fsync() on all Append() calls
I20260619 06:53:57.277180 1176 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:53:57.278191 1176 tablet_bootstrap.cc:492] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Bootstrap complete.
I20260619 06:53:57.278908 1176 ts_tablet_manager.cc:1397] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.008s
I20260619 06:53:57.286054 1120 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42631
I20260619 06:53:57.286149 1283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42631 every 8 connection(s)
I20260619 06:53:57.287743 1120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:53:57.290608 1176 raft_consensus.cc:348] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:57.291714 1176 raft_consensus.cc:729] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d18d485ca8417b8a587c341707f872, State: Initialized, Role: FOLLOWER
I20260619 06:53:57.292445 1176 consensus_queue.cc:260] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:57.293195 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.290752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:50111)
0619 06:53:57.291677 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.292057 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":419,"threads_started":1}
W20260619 06:53:57.293896 1284 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:41087 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:41087: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
I20260619 06:53:57.293996 1176 ts_tablet_manager.cc:1428] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260619 06:53:57.294816 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.294481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:33317)
0619 06:53:57.294633 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.294705 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 06:53:57.295094 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1120
I20260619 06:53:57.295746 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:41747
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=39933
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:57.298256 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.297882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:38425)
0619 06:53:57.298040 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.298129 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 06:53:57.298491 1284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:53:57.633755 1288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:57.634059 1288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:57.634253 1288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:57.641695 1288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:57.641844 1288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:57.642035 1288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:53:57.652009 1288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:41747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=39933
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:57.653360 1288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:57.654708 1288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:57.667801 1294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:57.667824 1296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:57.667848 1293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:57.668901 1288 server_base.cc:1034] running on GCE node
I20260619 06:53:57.669628 1288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:57.670562 1288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:57.671790 1288 hybrid_clock.cc:648] HybridClock initialized: now 1781852037671745 us; error 57 us; skew 500 ppm
I20260619 06:53:57.672127 1288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:57 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852037, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:57.675268 1288 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:53:57.676738 1288 webserver.cc:460] Webserver started at http://127.18.39.66:39933/ using document root <none> and password file <none>
I20260619 06:53:57.677507 1288 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:57.677655 1288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:57.683097 1288 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:53:57.686306 1303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:57.687759 1288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:53:57.687983 1288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "5d8d9be338e8498b845bad51c017ec31"
format_stamp: "Formatted at 2026-06-19 06:53:51 on dist-test-slave-1h5s"
server_key: "1f03b2ccf9a7829737d53fd070363219"
server_key_iv: "9a6db13588fcfd5d77f4f987862c9f8d"
server_key_version: "encryptionkey@0"
I20260619 06:53:57.689015 1288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:57.706357 1288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:57.710897 1288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:57.711196 1288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:57.712203 1288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:57.713589 1288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:57.713721 1288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:57.713850 1288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:57.713945 1288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:57.747676 1288 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:41747
I20260619 06:53:57.747763 1415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:41747 every 8 connection(s)
I20260619 06:53:57.749258 1288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:53:57.749632 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1288
I20260619 06:53:57.750437 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:44775
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:41087
--webserver_port=43083
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:53:57.754943 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.752550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:54989)
0619 06:53:57.753435 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.753697 (+ 262us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":315,"threads_started":1}
W20260619 06:53:57.755729 1416 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:41087 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:41087: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
I20260619 06:53:57.756757 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.756288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:35949)
0619 06:53:57.756491 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.756588 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:53:57.757956 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:57.757496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:37331)
0619 06:53:57.757673 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:57.757780 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:53:57.758231 1416 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:53:58.084149 1419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:53:58.084445 1419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:53:58.084636 1419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:53:58.092113 1419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:53:58.092281 1419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:53:58.092501 1419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:53:58.102916 1419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:44775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=43083
--tserver_master_addrs=127.18.39.126:41087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:53:58.104319 1419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:53:58.105666 1419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:53:58.119238 1424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:58.119279 1427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:53:58.119202 1425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:53:58.120113 1419 server_base.cc:1034] running on GCE node
I20260619 06:53:58.120703 1419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:53:58.121703 1419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:53:58.122931 1419 hybrid_clock.cc:648] HybridClock initialized: now 1781852038122852 us; error 42 us; skew 500 ppm
I20260619 06:53:58.123271 1419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:53:58 dist-test-slave-1h5s krb5kdc[32439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852038, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:53:58.126559 1419 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:53:58.127947 1419 webserver.cc:460] Webserver started at http://127.18.39.67:43083/ using document root <none> and password file <none>
I20260619 06:53:58.128602 1419 fs_manager.cc:359] Metadata directory not provided
I20260619 06:53:58.128702 1419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:53:58.134141 1419 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:53:58.136729 1434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:58.138178 1419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:53:58.138312 1419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "eca9a8e0bf5541539222fdb2b9651e7c"
format_stamp: "Formatted at 2026-06-19 06:53:52 on dist-test-slave-1h5s"
server_key: "b10b08fb0df403aa447a2d789c7d87cc"
server_key_iv: "6f4206d2bfed8aaff2e96c641d1844b4"
server_key_version: "encryptionkey@0"
I20260619 06:53:58.139242 1419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:53:58.160071 1419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:53:58.164546 1419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:53:58.164768 1419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:53:58.165704 1419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:53:58.166988 1419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:53:58.167075 1419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:58.167153 1419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:53:58.167207 1419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:53:58.201728 1419 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44775
I20260619 06:53:58.201800 1546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44775 every 8 connection(s)
I20260619 06:53:58.203241 1419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:53:58.205564 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1419
I20260619 06:53:58.208357 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.206371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:37023)
0619 06:53:58.207191 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.207438 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":343,"threads_started":1}
W20260619 06:53:58.209022 1547 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:41087 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:41087: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
I20260619 06:53:58.209966 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.209622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:39239)
0619 06:53:58.209761 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.209862 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 06:53:58.210868 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.210452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:51533)
0619 06:53:58.210585 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.210719 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:53:58.211117 1547 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:53:58.299711 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.298976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:46671)
0619 06:53:58.299405 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.299521 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":209,"threads_started":1}
I20260619 06:53:58.723176 1552 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:53:58.723523 1552 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:53:58.725240 1552 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
Jun 19 06:53:58 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852037, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
Jun 19 06:53:58 dist-test-slave-1h5s krb5kdc[32439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852037, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:53:58.752172 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.726314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:41747 (local address 127.18.39.65:56765)
0619 06:53:58.727016 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.727033 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:58.727150 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:58.730467 (+ 3317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:58.730472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:58.731227 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:58.731802 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:58.731841 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.734169 (+ 2328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.734174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:58.736590 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:58.736606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.738069 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.738074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:58.738428 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:58.739371 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:58.739402 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:58.742598 (+ 3196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:58.747654 (+ 5056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:58.747658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:58.747671 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:58.748515 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:58.749126 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:58.749131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:58.749135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:58.749428 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:58.750461 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:58.750468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:58.750823 (+ 355us) client_negotiation.cc:769] Sending connection context
0619 06:53:58.751154 (+ 331us) client_negotiation.cc:240] Negotiation successful
0619 06:53:58.751697 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":296,"spinlock_wait_cycles":29568,"thread_start_us":718,"threads_started":1}
I20260619 06:53:58.752380 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.725798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:52215 (local address 127.18.39.67:44775)
0619 06:53:58.726556 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:58.726568 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:58.726611 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:53:58.729002 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:58.729014 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:58.729532 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:58.729911 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:58.732035 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.733938 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.736767 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.737842 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.738030 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:58.743182 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:58.743195 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:58.743213 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:58.743262 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:58.747402 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:58.749450 (+ 2048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:58.749462 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:58.749481 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:58.749686 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:58.750081 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:58.750087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:58.750090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:58.750696 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:58.750875 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:58.751552 (+ 677us) server_negotiation.cc:299] Negotiation successful
0619 06:53:58.751950 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":471,"threads_started":1}
I20260619 06:53:58.752556 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.726971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:56765 (local address 127.18.39.66:41747)
0619 06:53:58.727560 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0619 06:53:58.727574 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:53:58.727611 (+ 37us) server_negotiation.cc:407] Connection header received
0619 06:53:58.729728 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:53:58.729741 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:53:58.730162 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:53:58.730486 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:53:58.732035 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.733938 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.736767 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.737842 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.738030 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:58.742953 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:53:58.742989 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:53:58.743012 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:53:58.743074 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:53:58.747479 (+ 4405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:58.748668 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:58.748678 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:58.748686 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:58.748950 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:53:58.749558 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:53:58.749562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:53:58.749565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:53:58.750292 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:53:58.750523 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 06:53:58.751379 (+ 856us) server_negotiation.cc:299] Negotiation successful
0619 06:53:58.751833 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":301,"threads_started":1}
I20260619 06:53:58.753206 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.725830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44775 (local address 127.18.39.65:52215)
0619 06:53:58.726087 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.726122 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:53:58.726688 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:53:58.729986 (+ 3298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:53:58.729994 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:53:58.730830 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:53:58.731802 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:58.731841 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.734171 (+ 2330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.734175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:58.736589 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:53:58.736606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:53:58.738057 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:53:58.738064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:53:58.738424 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:53:58.739371 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:53:58.739414 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:53:58.742903 (+ 3489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:53:58.747643 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:58.747654 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:58.747672 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:58.749043 (+ 1371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:58.749812 (+ 769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:53:58.749817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:53:58.749820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:53:58.749958 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:53:58.750950 (+ 992us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:53:58.750955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:53:58.751105 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 06:53:58.751232 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 06:53:58.752946 (+ 1714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":1710,"spinlock_wait_cycles":23296}
W20260619 06:53:58.755677 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
W20260619 06:53:58.756078 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
I20260619 06:53:58.756191 1171 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16d18d485ca8417b8a587c341707f872; no voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c
I20260619 06:53:58.756536 1552 raft_consensus.cc:2726] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:53:58.759575 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:58.758689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:42931)
0619 06:53:58.759131 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:58.759266 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":198,"threads_started":1}
I20260619 06:53:59.212452 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:59.211612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:37095)
0619 06:53:59.212070 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:59.212225 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":218,"threads_started":1}
I20260619 06:53:59.301287 1558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:59.300612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:45733)
0619 06:53:59.301044 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:59.301125 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":206,"threads_started":1}
I20260619 06:53:59.761270 1559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:53:59.760483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:41807)
0619 06:53:59.760972 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 06:53:59.761081 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":210,"threads_started":1}
I20260619 06:54:00.214069 1560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:00.213315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:54431)
0619 06:54:00.213827 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:00.213908 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":253,"threads_started":1}
I20260619 06:54:00.303056 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:00.302255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:45937)
0619 06:54:00.302737 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:00.302835 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":249,"threads_started":1}
I20260619 06:54:00.723978 1562 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:00.724217 1562 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:54:00.724865 1562 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
W20260619 06:54:00.725999 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
W20260619 06:54:00.726244 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
I20260619 06:54:00.726408 1171 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16d18d485ca8417b8a587c341707f872; no voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c
I20260619 06:54:00.726672 1562 raft_consensus.cc:2726] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:54:00.763072 1563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:00.762298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:35967)
0619 06:54:00.762809 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:00.762903 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260619 06:54:01.215768 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:01.215048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:44321)
0619 06:54:01.215516 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:01.215606 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260619 06:54:01.304842 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:01.304040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:45331)
0619 06:54:01.304577 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:01.304677 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":238,"threads_started":1}
I20260619 06:54:01.765267 1566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:01.764270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:32789)
0619 06:54:01.764916 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:01.765019 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":354,"threads_started":1}
I20260619 06:54:02.217842 1567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:02.216860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:35063)
0619 06:54:02.217502 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:02.217607 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":302,"threads_started":1}
I20260619 06:54:02.307337 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:02.306363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:45937)
0619 06:54:02.306975 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:02.307083 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":296,"threads_started":1}
I20260619 06:54:02.636170 1569 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:02.636394 1569 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:54:02.637043 1569 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
W20260619 06:54:02.638347 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
W20260619 06:54:02.638645 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
I20260619 06:54:02.638796 1171 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16d18d485ca8417b8a587c341707f872; no voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c
I20260619 06:54:02.639088 1569 raft_consensus.cc:2726] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:54:02.767251 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:02.766427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:43363)
0619 06:54:02.766951 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:02.767056 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":249,"threads_started":1}
I20260619 06:54:03.219645 1571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:03.218880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:59793)
0619 06:54:03.219354 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:03.219476 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260619 06:54:03.309197 1572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:03.308428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:36115)
0619 06:54:03.308881 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:03.309008 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":226,"threads_started":1}
I20260619 06:54:03.769232 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:03.768297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:54639)
0619 06:54:03.768864 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:03.768987 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":298,"threads_started":1}
I20260619 06:54:04.221271 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:04.220545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:33659)
0619 06:54:04.221010 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:04.221092 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260619 06:54:04.310994 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:04.310230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:59063)
0619 06:54:04.310672 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:04.310764 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":244,"threads_started":1}
I20260619 06:54:04.728665 1576 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:04.728931 1576 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:54:04.729691 1576 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
W20260619 06:54:04.730573 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
W20260619 06:54:04.730804 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
I20260619 06:54:04.730890 1171 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16d18d485ca8417b8a587c341707f872; no voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c
I20260619 06:54:04.731159 1576 raft_consensus.cc:2726] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:54:04.771113 1577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:04.770320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:52511)
0619 06:54:04.770829 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:04.770907 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":245,"threads_started":1}
I20260619 06:54:05.223299 1578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:05.222404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:45389)
0619 06:54:05.223020 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:05.223112 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":303,"threads_started":1}
I20260619 06:54:05.312682 1579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:05.311966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:42671)
0619 06:54:05.312428 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:05.312518 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":245,"threads_started":1}
I20260619 06:54:05.772886 1580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:05.772114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:50029)
0619 06:54:05.772615 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:05.772698 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":285,"threads_started":1}
I20260619 06:54:06.225062 1581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:06.224302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:49321)
0619 06:54:06.224756 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:06.224842 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":248,"threads_started":1}
I20260619 06:54:06.314338 1582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:06.313653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:41055)
0619 06:54:06.314082 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:06.314168 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":210,"threads_started":1}
I20260619 06:54:06.774927 1583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:06.774062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:38947)
0619 06:54:06.774656 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:06.774763 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":303,"threads_started":1}
I20260619 06:54:07.226697 1584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:07.225963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:48769)
0619 06:54:07.226449 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:07.226531 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":238,"threads_started":1}
I20260619 06:54:07.316237 1585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:07.315390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:58219)
0619 06:54:07.315971 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:07.316069 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":234,"threads_started":1}
I20260619 06:54:07.776738 1586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:07.775892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:43175)
0619 06:54:07.776414 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:07.776539 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":268,"threads_started":1}
I20260619 06:54:08.228396 1587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:08.227685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:54065)
0619 06:54:08.228138 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:08.228221 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":248,"threads_started":1}
I20260619 06:54:08.318126 1588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:08.317298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:51667)
0619 06:54:08.317827 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:08.317911 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":296,"threads_started":1}
I20260619 06:54:08.778620 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:08.777815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:32807)
0619 06:54:08.778327 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:08.778425 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":262,"threads_started":1}
I20260619 06:54:09.230203 1590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:09.229434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:34965)
0619 06:54:09.229942 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:09.230038 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":253,"threads_started":1}
I20260619 06:54:09.319868 1591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:09.319098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:37317)
0619 06:54:09.319591 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:09.319687 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":241,"threads_started":1}
I20260619 06:54:09.780360 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:09.779593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:46929)
0619 06:54:09.780088 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:09.780178 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":248,"threads_started":1}
I20260619 06:54:10.231904 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:10.231192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:53849)
0619 06:54:10.231641 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:10.231725 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":233,"threads_started":1}
I20260619 06:54:10.321856 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:10.320942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:37615)
0619 06:54:10.321566 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:10.321670 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":317,"threads_started":1}
I20260619 06:54:10.782588 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:10.781654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:32795)
0619 06:54:10.782205 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:10.782345 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":271,"threads_started":1}
I20260619 06:54:11.233939 1596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:11.233045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:59613)
0619 06:54:11.233612 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:11.233731 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":303,"threads_started":1}
I20260619 06:54:11.323861 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:11.322987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:52623)
0619 06:54:11.323520 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:11.323617 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":293,"threads_started":1}
I20260619 06:54:11.784685 1598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:11.783770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:47491)
0619 06:54:11.784350 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:11.784448 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":299,"threads_started":1}
I20260619 06:54:12.020967 1599 raft_consensus.cc:482] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:12.021229 1599 raft_consensus.cc:504] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eca9a8e0bf5541539222fdb2b9651e7c" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44775 } } peers { permanent_uuid: "16d18d485ca8417b8a587c341707f872" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42631 } } peers { permanent_uuid: "5d8d9be338e8498b845bad51c017ec31" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 41747 } }
I20260619 06:54:12.021936 1599 leader_election.cc:288] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775), 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747)
W20260619 06:54:12.023074 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eca9a8e0bf5541539222fdb2b9651e7c (127.18.39.67:44775): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
W20260619 06:54:12.023314 1171 leader_election.cc:341] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5d8d9be338e8498b845bad51c017ec31 (127.18.39.66:41747): Not found: Tablet not found: 835b50ef051442929c892f2cac670a58
I20260619 06:54:12.023444 1171 leader_election.cc:302] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16d18d485ca8417b8a587c341707f872; no voters: 5d8d9be338e8498b845bad51c017ec31, eca9a8e0bf5541539222fdb2b9651e7c
I20260619 06:54:12.023756 1599 raft_consensus.cc:2726] T 835b50ef051442929c892f2cac670a58 P 16d18d485ca8417b8a587c341707f872 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:54:12.235780 1600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:12.234958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:32885)
0619 06:54:12.235446 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:12.235565 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":289,"threads_started":1}
I20260619 06:54:12.326036 1601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:12.325129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.65:59119)
0619 06:54:12.325681 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:12.325805 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":275,"threads_started":1}
I20260619 06:54:12.786981 1602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:12.786082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.66:51437)
0619 06:54:12.786695 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:12.786812 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":328,"threads_started":1}
I20260619 06:54:13.215621 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1120
I20260619 06:54:13.235441 1603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:13.217052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.18.39.65:42631)
0619 06:54:13.217610 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:13.217619 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:13.217658 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:54:13.217723 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:13.217730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:13.217952 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:13.218104 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:13.219737 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.220804 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.222249 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.222789 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.222907 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:13.228993 (+ 6086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:13.229010 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:13.229069 (+ 59us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:13.229114 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:13.232444 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.233276 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.233285 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.233287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.233639 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.234005 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.234009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.234011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.234386 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:13.234660 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:13.234912 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 06:54:13.235146 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":300,"threads_started":1}
I20260619 06:54:13.236845 1279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:13.238397 1605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:13.236979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:41087 (local address 127.18.39.67:46489)
0619 06:54:13.237924 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:13.238087 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:41087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":535,"threads_started":1}
I20260619 06:54:13.330518 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1120
I20260619 06:54:13.349800 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1288
I20260619 06:54:13.364430 1607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:13.350627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.18.39.66:41747)
0619 06:54:13.351175 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:13.351181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:13.351198 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:54:13.351255 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:13.351259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:13.351320 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:13.351448 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:13.353071 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.353921 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.355222 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.355671 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.355843 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:13.358963 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:13.358968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:13.358972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:13.359012 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:13.361616 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.362356 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.362362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.362365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.362524 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.362914 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.362920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.362923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.363217 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:13.363495 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:13.363798 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 06:54:13.364040 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":263,"threads_started":1}
I20260619 06:54:13.365249 1411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:13.454342 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1288
I20260619 06:54:13.474797 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1419
I20260619 06:54:13.489387 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:13.475749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40126 (local address 127.18.39.67:44775)
0619 06:54:13.476339 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:13.476343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:13.476361 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:54:13.476417 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:13.476421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:13.476489 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:13.476627 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:13.478298 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.479073 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.480594 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:13.481073 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:13.481216 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:13.483896 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:13.483901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:13.483904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:13.483937 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:13.486527 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.487286 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.487295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.487298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.487443 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:13.487831 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:13.487837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:13.487841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:13.488110 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:13.488399 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:13.488696 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 06:54:13.488929 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":330,"threads_started":1}
I20260619 06:54:13.490170 1542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:13.575268 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1419
2026-06-19T06:54:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24499 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:54:13 dist-test-slave-1h5s krb5kdc[1613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:54:13 dist-test-slave-1h5s krb5kdc[1613](info): set up 2 sockets
Jun 19 06:54:13 dist-test-slave-1h5s krb5kdc[1613](info): commencing operation
krb5kdc: starting...
W20260619 06:54:14.690456 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:54:14 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:54:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:54:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:54:14.935108 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:54:15.273490 1626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:15.273864 1626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:15.274044 1626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:15.281565 1626 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:54:15.281704 1626 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:54:15.281773 1626 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:54:15.281872 1626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:54:15.281955 1626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:15.282085 1626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:15.282176 1626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:54:15.282256 1626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:54:15.292570 1626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:15.293982 1626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:15.295379 1626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:15.308032 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:15.308056 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:15.307992 1634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:15.308763 1626 server_base.cc:1034] running on GCE node
I20260619 06:54:15.309759 1626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:15.311317 1626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:15.312549 1626 hybrid_clock.cc:648] HybridClock initialized: now 1781852055312491 us; error 38 us; skew 500 ppm
I20260619 06:54:15.312933 1626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:15 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852055, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:15.316335 1626 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:54:15.317996 1626 webserver.cc:460] Webserver started at http://127.18.39.126:43419/ using document root <none> and password file <none>
I20260619 06:54:15.318730 1626 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:15.318819 1626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:15.319154 1626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:54:15.322343 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "076f444fe6339b01a57611f055282507"
server_key_iv: "9205c24885c60ebf4530a7d10abe59be"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.323194 1626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "076f444fe6339b01a57611f055282507"
server_key_iv: "9205c24885c60ebf4530a7d10abe59be"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.328792 1626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:54:15.332631 1641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:15.334241 1626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:54:15.334452 1626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "076f444fe6339b01a57611f055282507"
server_key_iv: "9205c24885c60ebf4530a7d10abe59be"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.334704 1626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:15.370944 1626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:15.375713 1626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:15.376030 1626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:15.395849 1626 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35971
I20260619 06:54:15.395877 1692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35971 every 8 connection(s)
I20260619 06:54:15.397357 1626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:54:15.399557 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1626
I20260619 06:54:15.399780 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:54:15.400655 18589 external_mini_cluster.cc:1428] Setting key 2d456e65cc19b12b8f5c3bda7f020f2d
I20260619 06:54:15.402104 1693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:54:15 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:15.418190 1693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap starting.
I20260619 06:54:15.421849 1693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:15.423077 1693 log.cc:824] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:15.426748 1693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: No bootstrap required, opened a new log
I20260619 06:54:15.427330 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:15.402855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.18.39.126:35971)
0619 06:54:15.403678 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:15.403693 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:15.403746 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:54:15.405838 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:15.405870 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:15.406451 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:15.406936 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:15.408343 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:15.410459 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:15.412100 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:15.413415 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:15.413677 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:15.419410 (+ 5733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:15.419418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:15.419434 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:15.419473 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:15.422689 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:15.423625 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:15.423632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:15.423659 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:15.423919 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:15.424318 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:15.424324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:15.424328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:15.424963 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:15.425197 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:15.425835 (+ 638us) server_negotiation.cc:299] Negotiation successful
0619 06:54:15.426232 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":330,"threads_started":1}
I20260619 06:54:15.434828 1693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:15.435261 1693 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:15.435392 1693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Initialized, Role: FOLLOWER
I20260619 06:54:15.435988 1693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:15.436223 1693 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:54:15.436362 1693 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:54:15.436543 1693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:15.439029 1693 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:15.439610 1693 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43ddb2ef8b4076bf0e2fac99b78d27; no voters:
I20260619 06:54:15.440217 1693 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:54:15.440392 1698 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:54:15.441072 1698 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 LEADER]: Becoming Leader. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Running, Role: LEADER
I20260619 06:54:15.441663 1698 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:15.442353 1693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:54:15.444312 1700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43ddb2ef8b4076bf0e2fac99b78d27. Latest consensus state: current_term: 1 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:54:15.444666 1700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:54:15.445830 1699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:54:15.446485 1699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:54:15.447685 1704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:54:15.451911 1704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:54:15.462344 1704 catalog_manager.cc:1347] Generated new cluster ID: 9a72e37dd8d74150b40bc15040ab8e86
I20260619 06:54:15.462471 1704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:54:15.500682 1704 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:54:15.502465 1704 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:54:15.510552 1704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Generated new TSK 0
I20260619 06:54:15.511512 1704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:54:15.577862 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:54:15.937225 1719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:15.937593 1719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:15.937783 1719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:15.945650 1719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:15.945757 1719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:15.945952 1719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:54:15.956135 1719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:15.957460 1719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:15.958748 1719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:15.972251 1725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:15.972332 1727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:15.972354 1724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:15.973196 1719 server_base.cc:1034] running on GCE node
I20260619 06:54:15.973915 1719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:15.974951 1719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:15.976217 1719 hybrid_clock.cc:648] HybridClock initialized: now 1781852055976168 us; error 57 us; skew 500 ppm
I20260619 06:54:15.976564 1719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:15 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852055, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:15.979851 1719 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:54:15.981392 1719 webserver.cc:460] Webserver started at http://127.18.39.65:36011/ using document root <none> and password file <none>
I20260619 06:54:15.982102 1719 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:15.982249 1719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:15.982564 1719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:54:15.985594 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "0c6ecc1898854cabb58386b79ecfe12f"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "34f7fdf6064668eae1a8a205115ebb26"
server_key_iv: "f9be742e2bedd774bfa715fb1a3549e4"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.986388 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "0c6ecc1898854cabb58386b79ecfe12f"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "34f7fdf6064668eae1a8a205115ebb26"
server_key_iv: "f9be742e2bedd774bfa715fb1a3549e4"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.991748 1719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:54:15.995482 1734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:15.996984 1719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:54:15.997210 1719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "0c6ecc1898854cabb58386b79ecfe12f"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "34f7fdf6064668eae1a8a205115ebb26"
server_key_iv: "f9be742e2bedd774bfa715fb1a3549e4"
server_key_version: "encryptionkey@0"
I20260619 06:54:15.997506 1719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:16.037761 1719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:16.042860 1719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:16.043185 1719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:16.044305 1719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:16.045768 1719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:16.045873 1719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:16.046027 1719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:16.046100 1719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:16.080403 1719 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42809
I20260619 06:54:16.080492 1846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42809 every 8 connection(s)
I20260619 06:54:16.081756 1719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:54:16.082854 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1719
I20260619 06:54:16.083058 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:54:16.083616 18589 external_mini_cluster.cc:1428] Setting key 1eddd7dc2c6c42c0cb82882f3b74910c
Jun 19 06:54:16 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852055, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:16.104465 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:16.084671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46375 (local address 127.18.39.126:35971)
0619 06:54:16.085122 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:16.085128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:16.086045 (+ 917us) server_negotiation.cc:407] Connection header received
0619 06:54:16.087893 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:16.087901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:16.087989 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:16.088109 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:16.089772 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.090996 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.092505 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.093060 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.093163 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:16.097411 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:16.097420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:16.097428 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:16.097474 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:16.101113 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:16.102025 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:16.102034 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:16.102039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:16.102200 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:16.102636 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:16.102640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:16.102643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:16.102928 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:16.103101 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:16.103829 (+ 728us) server_negotiation.cc:299] Negotiation successful
0619 06:54:16.104068 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260619 06:54:16.105603 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:16.085077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:46375)
0619 06:54:16.085877 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:16.085931 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:16.087631 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:16.088297 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:16.088305 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:16.088856 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:16.089605 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:16.089625 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.091142 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.091146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:16.092364 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:16.092375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.093302 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.093311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:16.093663 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:16.094415 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:16.094450 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:16.097141 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:16.101259 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:16.101267 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:16.101282 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:16.101894 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:16.102310 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:16.102313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:16.102315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:16.102500 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:16.103067 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:16.103076 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:16.103445 (+ 369us) client_negotiation.cc:769] Sending connection context
0619 06:54:16.103726 (+ 281us) client_negotiation.cc:240] Negotiation successful
0619 06:54:16.104293 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":339,"threads_started":1}
I20260619 06:54:16.107208 1847 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:16.107594 1847 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:16.108546 1847 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
I20260619 06:54:16.110869 1658 ts_manager.cc:194] Registered new tserver with Master: 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
I20260619 06:54:16.112881 1658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:46375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:54:16.140599 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:54:16.478227 1853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:16.478588 1853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:16.478765 1853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:16.486240 1853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:16.486399 1853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:16.486593 1853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:54:16.496763 1853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:16.498245 1853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:16.499630 1853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:16.513581 1858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:16.513823 1861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:16.513873 1859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:16.514954 1853 server_base.cc:1034] running on GCE node
I20260619 06:54:16.515640 1853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:16.516647 1853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:16.517854 1853 hybrid_clock.cc:648] HybridClock initialized: now 1781852056517816 us; error 33 us; skew 500 ppm
I20260619 06:54:16.518231 1853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:16 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852056, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:16.521481 1853 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:54:16.522954 1853 webserver.cc:460] Webserver started at http://127.18.39.66:36931/ using document root <none> and password file <none>
I20260619 06:54:16.523770 1853 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:16.523910 1853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:16.524261 1853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:54:16.527485 1853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
format_stamp: "Formatted at 2026-06-19 06:54:16 on dist-test-slave-1h5s"
server_key: "3f75d99f6570444cb936afa126a95e2a"
server_key_iv: "8492a1f73730b107af954c88af4555f2"
server_key_version: "encryptionkey@0"
I20260619 06:54:16.528331 1853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
format_stamp: "Formatted at 2026-06-19 06:54:16 on dist-test-slave-1h5s"
server_key: "3f75d99f6570444cb936afa126a95e2a"
server_key_iv: "8492a1f73730b107af954c88af4555f2"
server_key_version: "encryptionkey@0"
I20260619 06:54:16.534049 1853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:54:16.538012 1868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:16.539710 1853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:54:16.539873 1853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
format_stamp: "Formatted at 2026-06-19 06:54:16 on dist-test-slave-1h5s"
server_key: "3f75d99f6570444cb936afa126a95e2a"
server_key_iv: "8492a1f73730b107af954c88af4555f2"
server_key_version: "encryptionkey@0"
I20260619 06:54:16.540081 1853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:16.562609 1853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:16.567482 1853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:16.567780 1853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:16.568809 1853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:16.570369 1853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:16.570454 1853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:16.570562 1853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:16.570616 1853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:16.607010 1853 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:36667
I20260619 06:54:16.607065 1980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:36667 every 8 connection(s)
I20260619 06:54:16.608399 1853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:54:16.615438 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1853
I20260619 06:54:16.615660 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:54:16.616176 18589 external_mini_cluster.cc:1428] Setting key 155ff3b54f5a6e66931c858b0c837400
Jun 19 06:54:16 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852056, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:16.630020 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:16.611175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:53565 (local address 127.18.39.126:35971)
0619 06:54:16.611630 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:16.611635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:16.612603 (+ 968us) server_negotiation.cc:407] Connection header received
0619 06:54:16.614531 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:16.614538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:16.614634 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:16.614754 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:16.616462 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.617227 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.618831 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.619286 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.619425 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:16.623229 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:16.623235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:16.623238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:16.623271 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:16.626508 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:16.627521 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:16.627526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:16.627529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:16.627690 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:16.628187 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:16.628190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:16.628193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:16.628456 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:16.628580 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:16.629545 (+ 965us) server_negotiation.cc:299] Negotiation successful
0619 06:54:16.629727 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":244,"threads_started":1}
I20260619 06:54:16.631620 1984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:16.611697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:53565)
0619 06:54:16.612407 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:16.612456 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:16.614250 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:16.614998 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:16.615007 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:16.615630 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:16.616293 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:16.616312 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.617414 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.617419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:16.618670 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:16.618682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:16.619517 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:16.619526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:16.619772 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:16.620485 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:16.620512 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:16.622972 (+ 2460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:16.626665 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:16.626677 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:16.626699 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:16.627358 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:16.627807 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:16.627814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:16.627821 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:16.628074 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:16.628581 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:16.628592 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:16.629074 (+ 482us) client_negotiation.cc:769] Sending connection context
0619 06:54:16.629411 (+ 337us) client_negotiation.cc:240] Negotiation successful
0619 06:54:16.630024 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":289,"threads_started":1}
I20260619 06:54:16.633369 1981 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:16.633690 1981 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:16.634418 1981 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
I20260619 06:54:16.635989 1658 ts_manager.cc:194] Registered new tserver with Master: 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
I20260619 06:54:16.636988 1658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:53565
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:54:16.670534 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:54:16.996928 1987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:16.997232 1987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:16.997407 1987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:17.004887 1987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:17.005010 1987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:17.005247 1987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:54:17.015664 1987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:17.017058 1987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:17.018429 1987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:17.032368 1995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:17.032295 1993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:17.032338 1992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:17.033510 1987 server_base.cc:1034] running on GCE node
I20260619 06:54:17.034152 1987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:17.035157 1987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:17.036409 1987 hybrid_clock.cc:648] HybridClock initialized: now 1781852057036336 us; error 72 us; skew 500 ppm
I20260619 06:54:17.036774 1987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:17 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852057, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:17.040414 1987 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:54:17.042016 1987 webserver.cc:460] Webserver started at http://127.18.39.67:38947/ using document root <none> and password file <none>
I20260619 06:54:17.042783 1987 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:17.042927 1987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:17.043232 1987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:54:17.046365 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
format_stamp: "Formatted at 2026-06-19 06:54:17 on dist-test-slave-1h5s"
server_key: "f0ee6779a0e47de1d8b9fdee65fa4c2d"
server_key_iv: "87c21d928b57702db97e126430600d24"
server_key_version: "encryptionkey@0"
I20260619 06:54:17.047140 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
format_stamp: "Formatted at 2026-06-19 06:54:17 on dist-test-slave-1h5s"
server_key: "f0ee6779a0e47de1d8b9fdee65fa4c2d"
server_key_iv: "87c21d928b57702db97e126430600d24"
server_key_version: "encryptionkey@0"
I20260619 06:54:17.052790 1987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260619 06:54:17.056924 2002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:17.058465 1987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:54:17.058674 1987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
format_stamp: "Formatted at 2026-06-19 06:54:17 on dist-test-slave-1h5s"
server_key: "f0ee6779a0e47de1d8b9fdee65fa4c2d"
server_key_iv: "87c21d928b57702db97e126430600d24"
server_key_version: "encryptionkey@0"
I20260619 06:54:17.058904 1987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:17.074838 1987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:17.079488 1987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:17.079802 1987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:17.080840 1987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:17.082268 1987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:17.082369 1987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:17.082523 1987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:17.082594 1987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:17.115885 1987 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44871
I20260619 06:54:17.115885 2114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44871 every 8 connection(s)
I20260619 06:54:17.116120 1847 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:17.117470 1987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:54:17.124451 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 1987
I20260619 06:54:17.124651 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:54:17.125238 18589 external_mini_cluster.cc:1428] Setting key dac44d538ace57cbf293d7c44fd06607
Jun 19 06:54:17 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852057, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:17.138021 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.120195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:37555 (local address 127.18.39.126:35971)
0619 06:54:17.120384 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.120390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.121589 (+ 1199us) server_negotiation.cc:407] Connection header received
0619 06:54:17.123341 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.123345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.123425 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.123512 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:17.125486 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.126302 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.128344 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.128731 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.128827 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.132336 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:17.132346 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:17.132354 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:17.132400 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:17.134993 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.135758 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.135762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.135765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.135888 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.136275 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.136278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.136280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.136566 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:17.136691 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.137572 (+ 881us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.137749 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:54:17.139446 2117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.120688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:37555)
0619 06:54:17.121434 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.121477 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.123083 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.123728 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.123737 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.124453 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:17.125282 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.125310 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.126472 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.126477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.128187 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.128204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.128912 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.128916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.129106 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.129843 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:17.129866 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:17.132098 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:17.135159 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:17.135167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:17.135182 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:17.135626 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:17.135980 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:17.135983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:17.135985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:17.136169 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:17.136716 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:17.136722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:17.137132 (+ 410us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.137464 (+ 332us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.137910 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":265,"threads_started":1}
I20260619 06:54:17.140949 2115 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:17.141316 2115 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:17.142130 2115 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
I20260619 06:54:17.143783 1658 ts_manager.cc:194] Registered new tserver with Master: 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871)
I20260619 06:54:17.144826 1658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:37555
I20260619 06:54:17.153254 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:54:17.170857 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.157817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43516 (local address 127.18.39.126:35971)
0619 06:54:17.158008 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.158012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.158523 (+ 511us) server_negotiation.cc:407] Connection header received
0619 06:54:17.158678 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.158682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.158755 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.158843 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:17.160303 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.161034 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.162496 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.163012 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.163125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.165691 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:17.165698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:17.165704 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:17.165749 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:17.168082 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.168779 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.168784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.168788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.168917 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.169287 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.169294 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.169298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.169671 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:17.169955 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.170256 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.170487 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:54:17.174916 1658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:54:17.182188 1658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:54:17.205711 2127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.195050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:40168)
0619 06:54:17.196853 (+ 1803us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.196912 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.197132 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.197941 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.197953 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.197997 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:17.198520 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.198540 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.200535 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.200545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.202795 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.202809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.204664 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.204671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.204909 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.204961 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.205106 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.205298 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"thread_start_us":1302,"threads_started":1}
I20260619 06:54:17.205740 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.195265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40168 (local address 127.18.39.65:42809)
0619 06:54:17.195983 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.195996 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.197095 (+ 1099us) server_negotiation.cc:407] Connection header received
0619 06:54:17.197397 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.197409 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.197697 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.197914 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:17.198774 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.200359 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.202977 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.204440 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.204571 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.204821 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.205185 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.205379 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":348,"threads_started":1}
I20260619 06:54:17.207369 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.195592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.18.39.66:36667)
0619 06:54:17.197781 (+ 2189us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.197794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.199239 (+ 1445us) server_negotiation.cc:407] Connection header received
0619 06:54:17.199482 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.199494 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.199802 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.200012 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:17.200947 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.202473 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.204919 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.206287 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.206450 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.206670 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.206878 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.207037 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1977,"thread_start_us":1864,"threads_started":1}
I20260619 06:54:17.209810 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.198060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50904 (local address 127.18.39.67:44871)
0619 06:54:17.200899 (+ 2839us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.200909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.200938 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:54:17.201043 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.201050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.201270 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.201499 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:17.202253 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.203754 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.206092 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.208601 (+ 2509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.208724 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.208936 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.209449 (+ 513us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.209580 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2652,"thread_start_us":2509,"threads_started":1}
I20260619 06:54:17.209903 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.197602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44871 (local address 127.0.0.1:50904)
0619 06:54:17.198792 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.198810 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.198917 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.201508 (+ 2591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.201513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.201537 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:17.201940 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.202077 (+ 137us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.203884 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.203889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.205920 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.205936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.209066 (+ 3130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.209074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.209242 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.209275 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.209394 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.209538 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":1,"thread_start_us":707,"threads_started":1}
I20260619 06:54:17.210062 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.195971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36667 (local address 127.0.0.1:53308)
0619 06:54:17.198712 (+ 2741us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.198736 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.198889 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.200202 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.200206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.200230 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:17.200663 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.200679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.202623 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.202628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.204769 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.204790 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.206509 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.206515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.206689 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.206723 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.206822 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.206954 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2558,"mutex_wait_us":1,"thread_start_us":1267,"threads_started":1}
I20260619 06:54:17.210636 1782 tablet_service.cc:1467] Processing CreateTablet for tablet 1793b712283645aeb0895574261d1af6 (DEFAULT_TABLE table=test-table [id=ead2aae0560f41638055b829dcf52e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:17.212162 1782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1793b712283645aeb0895574261d1af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:17.213581 2050 tablet_service.cc:1467] Processing CreateTablet for tablet 1793b712283645aeb0895574261d1af6 (DEFAULT_TABLE table=test-table [id=ead2aae0560f41638055b829dcf52e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:17.214287 1916 tablet_service.cc:1467] Processing CreateTablet for tablet 1793b712283645aeb0895574261d1af6 (DEFAULT_TABLE table=test-table [id=ead2aae0560f41638055b829dcf52e64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:17.215070 2050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1793b712283645aeb0895574261d1af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:17.215579 1916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1793b712283645aeb0895574261d1af6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:17.225309 2133 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Bootstrap starting.
I20260619 06:54:17.230180 2134 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Bootstrap starting.
I20260619 06:54:17.230563 2133 tablet_bootstrap.cc:654] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:17.231195 2135 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Bootstrap starting.
I20260619 06:54:17.232062 2133 log.cc:824] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:17.234984 2133 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: No bootstrap required, opened a new log
I20260619 06:54:17.235268 2135 tablet_bootstrap.cc:654] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:17.235396 2133 ts_tablet_manager.cc:1397] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260619 06:54:17.236552 2134 tablet_bootstrap.cc:654] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:17.236804 2135 log.cc:824] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:17.237944 2134 log.cc:824] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:17.240304 2135 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: No bootstrap required, opened a new log
I20260619 06:54:17.240608 2135 ts_tablet_manager.cc:1397] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260619 06:54:17.241097 2134 tablet_bootstrap.cc:492] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: No bootstrap required, opened a new log
I20260619 06:54:17.241413 2134 ts_tablet_manager.cc:1397] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260619 06:54:17.244269 2133 raft_consensus.cc:348] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.244720 2133 raft_consensus.cc:374] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:17.244829 2133 raft_consensus.cc:729] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6ecc1898854cabb58386b79ecfe12f, State: Initialized, Role: FOLLOWER
I20260619 06:54:17.245535 2133 consensus_queue.cc:260] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.247488 2133 ts_tablet_manager.cc:1428] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260619 06:54:17.253151 2135 raft_consensus.cc:348] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.253727 2135 raft_consensus.cc:374] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:17.253850 2135 raft_consensus.cc:729] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a8c7e1341d46f2a75f8a639839bbe6, State: Initialized, Role: FOLLOWER
I20260619 06:54:17.254273 2134 raft_consensus.cc:348] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.254748 2134 raft_consensus.cc:374] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:17.254585 2135 consensus_queue.cc:260] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.254855 2134 raft_consensus.cc:729] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a5b9d6b429b4b75bddcd767ccff8af5, State: Initialized, Role: FOLLOWER
I20260619 06:54:17.255606 2134 consensus_queue.cc:260] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.255935 2115 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:17.256753 2135 ts_tablet_manager.cc:1428] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Time spent starting tablet: real 0.016s user 0.015s sys 0.002s
I20260619 06:54:17.256798 1981 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:17.257588 2134 ts_tablet_manager.cc:1428] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260619 06:54:17.283005 2139 raft_consensus.cc:482] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:17.283385 2139 raft_consensus.cc:504] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.285122 2139 leader_election.cc:288] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667), 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871)
I20260619 06:54:17.292418 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.286816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:37277 (local address 127.18.39.67:44871)
0619 06:54:17.287027 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.287034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.287300 (+ 266us) server_negotiation.cc:407] Connection header received
0619 06:54:17.287493 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.287499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.287563 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.287662 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:17.288203 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.288929 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.290701 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.291460 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.291657 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.291733 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.292007 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.292149 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:54:17.292557 2143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.286612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44871 (local address 127.18.39.65:37277)
0619 06:54:17.287185 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.287199 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.287351 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.287680 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.287684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.287707 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:17.288050 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.288058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.289067 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.289073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.290541 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.290553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.291673 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.291679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.291849 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.291876 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.291980 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.292134 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":281,"threads_started":1}
I20260619 06:54:17.292619 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.285678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:43169 (local address 127.18.39.66:36667)
0619 06:54:17.285877 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.285882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.286624 (+ 742us) server_negotiation.cc:407] Connection header received
0619 06:54:17.286996 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.287003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.287115 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.287175 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:17.287811 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.288602 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.290652 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.291522 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.291721 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.291809 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.292181 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.292322 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:54:17.292706 2142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.285623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36667 (local address 127.18.39.65:43169)
0619 06:54:17.286517 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:17.286539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:17.286764 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:17.287352 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:17.287356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:17.287378 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:17.287664 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.287671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.288749 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.288756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.290473 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:17.290486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.291779 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.291786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:17.291951 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.291975 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:54:17.292153 (+ 178us) client_negotiation.cc:240] Negotiation successful
0619 06:54:17.292300 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":422,"threads_started":1}
I20260619 06:54:17.293812 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1793b712283645aeb0895574261d1af6" candidate_uuid: "0c6ecc1898854cabb58386b79ecfe12f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" is_pre_election: true
I20260619 06:54:17.293823 2070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1793b712283645aeb0895574261d1af6" candidate_uuid: "0c6ecc1898854cabb58386b79ecfe12f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" is_pre_election: true
I20260619 06:54:17.294307 1936 raft_consensus.cc:2454] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6ecc1898854cabb58386b79ecfe12f in term 0.
I20260619 06:54:17.294339 2070 raft_consensus.cc:2454] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6ecc1898854cabb58386b79ecfe12f in term 0.
I20260619 06:54:17.294992 1738 leader_election.cc:302] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f, 8a5b9d6b429b4b75bddcd767ccff8af5; no voters:
I20260619 06:54:17.295444 2139 raft_consensus.cc:2781] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:54:17.295610 2139 raft_consensus.cc:482] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:54:17.295747 2139 raft_consensus.cc:3037] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:17.298072 2139 raft_consensus.cc:504] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.298612 2139 leader_election.cc:288] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 1 election: Requested vote from peers 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667), 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871)
I20260619 06:54:17.299095 1936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1793b712283645aeb0895574261d1af6" candidate_uuid: "0c6ecc1898854cabb58386b79ecfe12f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
I20260619 06:54:17.299193 2070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1793b712283645aeb0895574261d1af6" candidate_uuid: "0c6ecc1898854cabb58386b79ecfe12f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
I20260619 06:54:17.299314 1936 raft_consensus.cc:3037] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:17.299451 2070 raft_consensus.cc:3037] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:17.301752 1936 raft_consensus.cc:2454] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6ecc1898854cabb58386b79ecfe12f in term 1.
I20260619 06:54:17.302135 1738 leader_election.cc:302] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f, 8a5b9d6b429b4b75bddcd767ccff8af5; no voters:
I20260619 06:54:17.302266 2070 raft_consensus.cc:2454] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6ecc1898854cabb58386b79ecfe12f in term 1.
I20260619 06:54:17.302440 2139 raft_consensus.cc:2781] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:54:17.302975 2139 raft_consensus.cc:686] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 LEADER]: Becoming Leader. State: Replica: 0c6ecc1898854cabb58386b79ecfe12f, State: Running, Role: LEADER
I20260619 06:54:17.303462 2139 consensus_queue.cc:237] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } }
I20260619 06:54:17.308261 1657 catalog_manager.cc:5495] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65). New cstate: current_term: 1 leader_uuid: "0c6ecc1898854cabb58386b79ecfe12f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } health_report { overall_health: UNKNOWN } } }
I20260619 06:54:17.333665 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.328001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.18.39.65:42809)
0619 06:54:17.328246 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.328251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.328269 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:54:17.328369 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.328373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.328453 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.328605 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:54:17.329008 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.329897 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.331303 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.331735 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.331836 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.332335 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:54:17.332870 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:54:17.332981 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.333177 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.333254 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260619 06:54:17.334203 1848 tablet.cc:2369] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:54:17.342249 1936 raft_consensus.cc:1261] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Refusing update from remote peer 0c6ecc1898854cabb58386b79ecfe12f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:54:17.342267 2070 raft_consensus.cc:1261] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Refusing update from remote peer 0c6ecc1898854cabb58386b79ecfe12f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:54:17.343341 2144 consensus_queue.cc:1035] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:54:17.343751 2139 consensus_queue.cc:1035] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:54:17.357777 2148 mvcc.cc:205] Tried to move back new op lower bound from 7298466026862977024 to 7298466026718384128. Current Snapshot: MvccSnapshot[applied={T|T < 7298466026862977024}]
I20260619 06:54:17.373157 1657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:54:17.373507 1657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:54:17.377216 1657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Sending DeleteTablet for 3 replicas of tablet 1793b712283645aeb0895574261d1af6
I20260619 06:54:17.378597 1782 tablet_service.cc:1514] Processing DeleteTablet for tablet 1793b712283645aeb0895574261d1af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:17 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:40168
I20260619 06:54:17.378700 1916 tablet_service.cc:1514] Processing DeleteTablet for tablet 1793b712283645aeb0895574261d1af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:17 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:53308
I20260619 06:54:17.379251 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1719
I20260619 06:54:17.379513 2050 tablet_service.cc:1514] Processing DeleteTablet for tablet 1793b712283645aeb0895574261d1af6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:17 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:50904
I20260619 06:54:17.389581 2157 tablet_replica.cc:330] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: stopping tablet replica
I20260619 06:54:17.391113 2157 raft_consensus.cc:2229] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:54:17.393029 2157 raft_consensus.cc:2258] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:17.395648 2159 tablet_replica.cc:330] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: stopping tablet replica
I20260619 06:54:17.396186 2159 raft_consensus.cc:2229] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:54:17.396780 2159 raft_consensus.cc:2258] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:17.396943 2157 ts_tablet_manager.cc:1905] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:17.399137 2158 tablet_replica.cc:330] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: stopping tablet replica
I20260619 06:54:17.399667 2159 ts_tablet_manager.cc:1905] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:17.400983 2158 raft_consensus.cc:2229] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:54:17.401938 2158 raft_consensus.cc:2258] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:17.409523 2157 ts_tablet_manager.cc:1918] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:54:17.411167 2158 ts_tablet_manager.cc:1905] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:17.416520 2159 ts_tablet_manager.cc:1918] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:54:17.418196 2158 ts_tablet_manager.cc:1918] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:54:17.418498 2157 log.cc:1196] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/1793b712283645aeb0895574261d1af6
I20260619 06:54:17.419199 2157 ts_tablet_manager.cc:1939] T 1793b712283645aeb0895574261d1af6 P 0c6ecc1898854cabb58386b79ecfe12f: Deleting consensus metadata
Jun 19 06:54:17 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:54:17.421453 1642 catalog_manager.cc:4841] TS 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809): tablet 1793b712283645aeb0895574261d1af6 (table test-table [id=ead2aae0560f41638055b829dcf52e64]) successfully deleted
I20260619 06:54:17.421972 2158 log.cc:1196] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/1793b712283645aeb0895574261d1af6
I20260619 06:54:17.421989 2159 log.cc:1196] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/1793b712283645aeb0895574261d1af6
I20260619 06:54:17.422972 2158 ts_tablet_manager.cc:1939] T 1793b712283645aeb0895574261d1af6 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting consensus metadata
I20260619 06:54:17.423321 2159 ts_tablet_manager.cc:1939] T 1793b712283645aeb0895574261d1af6 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting consensus metadata
I20260619 06:54:17.425185 1645 catalog_manager.cc:4841] TS 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): tablet 1793b712283645aeb0895574261d1af6 (table test-table [id=ead2aae0560f41638055b829dcf52e64]) successfully deleted
I20260619 06:54:17.427831 1645 catalog_manager.cc:4841] TS 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): tablet 1793b712283645aeb0895574261d1af6 (table test-table [id=ead2aae0560f41638055b829dcf52e64]) successfully deleted
I20260619 06:54:17.431177 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.380438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.18.39.65:42809)
0619 06:54:17.380680 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.380689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.400919 (+ 20230us) server_negotiation.cc:407] Connection header received
0619 06:54:17.401287 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.401293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.401416 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.401508 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:17.405488 (+ 3980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.406877 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.413469 (+ 6592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.414214 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.414317 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.423588 (+ 9271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:17.423603 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:17.423622 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:17.423691 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:17.427917 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.428950 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.428958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.428960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.429123 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.429618 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.429622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.429624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.429980 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:17.430219 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.430655 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.430850 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 06:54:17.431877 1842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:17.534448 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1719
I20260619 06:54:17.554451 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1853
Jun 19 06:54:17 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:54:17.570072 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.555451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.18.39.66:36667)
0619 06:54:17.555621 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.555625 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.555645 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:54:17.555740 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.555744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.555805 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.555889 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:17.557380 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.558105 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.559472 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.559849 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.559941 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.564242 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:17.564277 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:17.564298 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:17.564353 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:17.567315 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.568041 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.568049 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.568052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.568174 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.568481 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.568484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.568486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.568872 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:17.569095 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.569647 (+ 552us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.569815 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:54:17.570880 1976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:17.661456 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1853
I20260619 06:54:17.678786 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1987
Jun 19 06:54:17 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:54:17.695670 2132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:17.679617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.18.39.67:44871)
0619 06:54:17.679819 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:17.679825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:17.679846 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:54:17.680020 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:17.680026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:17.680101 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:17.680241 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:17.681722 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.682473 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.683895 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:17.684319 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:17.684465 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:17.689436 (+ 4971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:17.689456 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:17.689474 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:17.689522 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:17.692512 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.693378 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.693391 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.693395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.693552 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:17.693970 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:17.693977 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:17.693980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:17.694365 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:17.694689 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:17.695038 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 06:54:17.695279 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:54:17.696404 2110 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:17.787705 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1987
I20260619 06:54:17.804390 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 1626
I20260619 06:54:17.805593 1688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:17.920493 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 1626
I20260619 06:54:37.942584 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--webserver_interface=127.18.39.126
--webserver_port=43419
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:54:38.327287 2165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:38.327641 2165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:38.327827 2165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:38.335572 2165 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:54:38.335692 2165 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:54:38.335769 2165 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:54:38.335846 2165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:54:38.335947 2165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:38.336040 2165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:38.336123 2165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:54:38.336220 2165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:54:38.346968 2165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=43419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:38.348649 2165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:38.350337 2165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:38.381626 2171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:38.381580 2172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:38.381935 2174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:38.383077 2165 server_base.cc:1034] running on GCE node
I20260619 06:54:38.384076 2165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:38.386067 2165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:38.387517 2165 hybrid_clock.cc:648] HybridClock initialized: now 1781852078387362 us; error 45 us; skew 500 ppm
I20260619 06:54:38.388286 2165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:38 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:38.392757 2165 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:54:38.394728 2165 webserver.cc:460] Webserver started at http://127.18.39.126:43419/ using document root <none> and password file <none>
I20260619 06:54:38.395651 2165 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:38.395777 2165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:38.403155 2165 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 06:54:38.406800 2181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:38.408569 2165 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260619 06:54:38.408788 2165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "076f444fe6339b01a57611f055282507"
server_key_iv: "9205c24885c60ebf4530a7d10abe59be"
server_key_version: "encryptionkey@0"
I20260619 06:54:38.410059 2165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:38.428612 2165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:38.433852 2165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:38.434187 2165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:38.455021 2165 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35971
I20260619 06:54:38.455065 2232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35971 every 8 connection(s)
I20260619 06:54:38.456844 2165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:54:38.463127 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2165
I20260619 06:54:38.464251 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:42809
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=36011
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:54:38.465634 2233 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:54:38.471380 2233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap starting.
I20260619 06:54:38.493489 2233 log.cc:824] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:38.506922 2233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:54:38.507545 2233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap complete.
I20260619 06:54:38.519543 2233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:38.520454 2233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Initialized, Role: FOLLOWER
I20260619 06:54:38.521273 2233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:38.521624 2233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:54:38.521754 2233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:54:38.521951 2233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:54:38.526615 2233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:38.527338 2233 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43ddb2ef8b4076bf0e2fac99b78d27; no voters:
I20260619 06:54:38.528033 2233 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:54:38.528184 2237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:54:38.529349 2237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 LEADER]: Becoming Leader. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Running, Role: LEADER
I20260619 06:54:38.530038 2237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:54:38.530593 2233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:54:38.532903 2239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43ddb2ef8b4076bf0e2fac99b78d27. Latest consensus state: current_term: 2 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:54:38.533421 2239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:54:38.538666 2238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:54:38.539285 2238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:54:38.541571 2242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:54:38.552994 2242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:54:38.555270 2242 catalog_manager.cc:1259] Loaded cluster ID: 9a72e37dd8d74150b40bc15040ab8e86
I20260619 06:54:38.555547 2242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:54:38.561245 2242 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:54:38.563639 2242 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Loaded TSK: 0
I20260619 06:54:38.619053 2242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Generated new TSK 1
I20260619 06:54:38.619614 2242 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:54:38.880034 2235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:38.880328 2235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:38.880486 2235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:38.887946 2235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:38.888069 2235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:38.888255 2235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:54:38.898859 2235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:42809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=36011
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:38.900218 2235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:38.901664 2235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:38.916749 2261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:38.917006 2260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:38.917414 2263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:38.923348 2235 server_base.cc:1034] running on GCE node
I20260619 06:54:38.924085 2235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:38.925222 2235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:38.926486 2235 hybrid_clock.cc:648] HybridClock initialized: now 1781852078926416 us; error 66 us; skew 500 ppm
I20260619 06:54:38.926841 2235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:38 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:38.930518 2235 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:54:38.932413 2235 webserver.cc:460] Webserver started at http://127.18.39.65:36011/ using document root <none> and password file <none>
I20260619 06:54:38.933254 2235 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:38.933439 2235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:38.939258 2235 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:54:38.942278 2270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:38.943806 2235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:54:38.943989 2235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "0c6ecc1898854cabb58386b79ecfe12f"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "34f7fdf6064668eae1a8a205115ebb26"
server_key_iv: "f9be742e2bedd774bfa715fb1a3549e4"
server_key_version: "encryptionkey@0"
I20260619 06:54:38.944998 2235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:38.974519 2235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:38.978938 2235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:38.979239 2235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:38.980291 2235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:38.981725 2235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:38.981843 2235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:38.981969 2235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:38.982056 2235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.015982 2235 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42809
I20260619 06:54:39.016040 2382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42809 every 8 connection(s)
I20260619 06:54:39.017601 2235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:54:39.024804 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2235
I20260619 06:54:39.025585 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:36667
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=36931
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:54:39 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852078, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:39.044461 2386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.020821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:49955)
0619 06:54:39.021621 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:39.021659 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:39.023399 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:39.024804 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:39.024816 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:39.025523 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:39.026151 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.026170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.028235 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.028241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.029588 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.029619 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.030655 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.030664 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.031000 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.031795 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:39.031822 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:39.034179 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:39.039216 (+ 5037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.039224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.039239 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.039970 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.040531 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.040537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.040543 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.040763 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.041692 (+ 929us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:39.041703 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:39.042070 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 06:54:39.042364 (+ 294us) client_negotiation.cc:240] Negotiation successful
0619 06:54:39.043013 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":317,"threads_started":1}
I20260619 06:54:39.044461 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.020592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:49955 (local address 127.18.39.126:35971)
0619 06:54:39.021193 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.021211 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.021784 (+ 573us) server_negotiation.cc:407] Connection header received
0619 06:54:39.023934 (+ 2150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.023946 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.024298 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.024664 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:39.026383 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.028039 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.029752 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.030393 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.030574 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.034561 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:39.034574 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:39.034599 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:39.034653 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:39.039006 (+ 4353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.040170 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.040181 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.040187 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.040392 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.040904 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.040917 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.040921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.041541 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:39.041762 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:39.042612 (+ 850us) server_negotiation.cc:299] Negotiation successful
0619 06:54:39.043031 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":270,"threads_started":1}
I20260619 06:54:39.047852 2383 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:39.048240 2383 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:39.049254 2383 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
I20260619 06:54:39.052034 2198 ts_manager.cc:194] Registered new tserver with Master: 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809)
I20260619 06:54:39.054175 2198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:49955
W20260619 06:54:39.359683 2387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:39.359988 2387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:39.360090 2387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:39.367669 2387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:39.367798 2387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:39.367913 2387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:54:39.378069 2387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:36667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=36931
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:39.379312 2387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:39.380524 2387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:39.393929 2392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:39.393992 2393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:39.393929 2395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:39.400482 2387 server_base.cc:1034] running on GCE node
I20260619 06:54:39.401170 2387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:39.402323 2387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:39.403563 2387 hybrid_clock.cc:648] HybridClock initialized: now 1781852079403492 us; error 79 us; skew 500 ppm
I20260619 06:54:39.403898 2387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:39 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:39.407182 2387 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:54:39.408900 2387 webserver.cc:460] Webserver started at http://127.18.39.66:36931/ using document root <none> and password file <none>
I20260619 06:54:39.409711 2387 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:39.409873 2387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:39.415640 2387 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 06:54:39.418416 2402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.419764 2387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:54:39.419942 2387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
format_stamp: "Formatted at 2026-06-19 06:54:16 on dist-test-slave-1h5s"
server_key: "3f75d99f6570444cb936afa126a95e2a"
server_key_iv: "8492a1f73730b107af954c88af4555f2"
server_key_version: "encryptionkey@0"
I20260619 06:54:39.420930 2387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:39.447046 2387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:39.451536 2387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:39.451768 2387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:39.452759 2387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:39.454130 2387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:39.454221 2387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.454365 2387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:39.454464 2387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.488829 2387 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:36667
I20260619 06:54:39.488920 2514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:36667 every 8 connection(s)
I20260619 06:54:39.490325 2387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:54:39.492261 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2387
I20260619 06:54:39.492957 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:44871
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=38947
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:54:39 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:39.512230 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.493020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:57187 (local address 127.18.39.126:35971)
0619 06:54:39.493245 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.493251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.494627 (+ 1376us) server_negotiation.cc:407] Connection header received
0619 06:54:39.496681 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.496688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.496817 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.496945 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:39.499271 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.500056 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.501860 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.502282 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.502433 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.506200 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:39.506206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:39.506209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:39.506248 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:39.508871 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.509837 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.509842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.509844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.509960 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.510406 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.510411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.510416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.510706 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:39.510875 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:39.511618 (+ 743us) server_negotiation.cc:299] Negotiation successful
0619 06:54:39.511881 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:54:39.513188 2517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.493615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:57187)
0619 06:54:39.494452 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:39.494499 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:39.496362 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:39.497210 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:39.497264 (+ 54us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:39.498202 (+ 938us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:39.499044 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.499068 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.500238 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.500244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.501684 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.501699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.502557 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.502565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.502827 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.503568 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:39.503598 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:39.505901 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:39.509042 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.509052 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.509072 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.509679 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.510071 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.510076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.510079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.510285 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.510846 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:39.510853 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:39.511240 (+ 387us) client_negotiation.cc:769] Sending connection context
0619 06:54:39.511525 (+ 285us) client_negotiation.cc:240] Negotiation successful
0619 06:54:39.512075 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":343,"threads_started":1}
I20260619 06:54:39.514693 2515 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:39.515061 2515 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:39.516045 2515 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
I20260619 06:54:39.518189 2198 ts_manager.cc:194] Registered new tserver with Master: 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
I20260619 06:54:39.519361 2198 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:57187
W20260619 06:54:39.829213 2518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:54:39.829627 2518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:54:39.829787 2518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:54:39.837901 2518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:54:39.838056 2518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:54:39.838263 2518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:54:39.848248 2518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:44871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=38947
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:54:39.849769 2518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:54:39.851079 2518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:54:39.864682 2523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:39.864645 2524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:54:39.864607 2526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:54:39.865685 2518 server_base.cc:1034] running on GCE node
I20260619 06:54:39.866299 2518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:54:39.867209 2518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:54:39.868419 2518 hybrid_clock.cc:648] HybridClock initialized: now 1781852079868373 us; error 49 us; skew 500 ppm
I20260619 06:54:39.868703 2518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:54:39 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:54:39.871877 2518 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:54:39.873486 2518 webserver.cc:460] Webserver started at http://127.18.39.67:38947/ using document root <none> and password file <none>
I20260619 06:54:39.874179 2518 fs_manager.cc:359] Metadata directory not provided
I20260619 06:54:39.874334 2518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:54:39.879782 2518 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:54:39.882778 2533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.884176 2518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:54:39.884384 2518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
format_stamp: "Formatted at 2026-06-19 06:54:17 on dist-test-slave-1h5s"
server_key: "f0ee6779a0e47de1d8b9fdee65fa4c2d"
server_key_iv: "87c21d928b57702db97e126430600d24"
server_key_version: "encryptionkey@0"
I20260619 06:54:39.885442 2518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:54:39.899186 2518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:54:39.903779 2518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:54:39.904085 2518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:54:39.905090 2518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:54:39.906471 2518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:54:39.906595 2518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.906719 2518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:54:39.906821 2518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:54:39.940626 2518 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44871
I20260619 06:54:39.940678 2645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44871 every 8 connection(s)
I20260619 06:54:39.942358 2518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:54:39.947945 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2518
Jun 19 06:54:39 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852079, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:54:39.964790 2385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.945021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:46125 (local address 127.18.39.126:35971)
0619 06:54:39.945203 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.945207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.946447 (+ 1240us) server_negotiation.cc:407] Connection header received
0619 06:54:39.948392 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.948398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.948489 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.948601 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:39.950615 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.951437 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.953753 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.954358 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.954448 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.958518 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:39.958523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:39.958526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:39.958556 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:39.961662 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.962439 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.962443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.962446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.962561 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.963004 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.963009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.963014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.963281 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:39.963443 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:39.964168 (+ 725us) server_negotiation.cc:299] Negotiation successful
0619 06:54:39.964402 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:54:39.965155 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.949034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.18.39.126:35971)
0619 06:54:39.950839 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.950843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.950856 (+ 13us) server_negotiation.cc:407] Connection header received
0619 06:54:39.950895 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.950898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.950946 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.951065 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:39.953049 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.953966 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.955950 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.956388 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.956484 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.959450 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:39.959455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:39.959458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:39.959501 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:39.962400 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.963095 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.963102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.963110 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.963220 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:39.963601 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:39.963607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:39.963611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:39.963883 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:39.964131 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:39.964478 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:54:39.964694 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1705,"thread_start_us":1599,"threads_started":1}
I20260619 06:54:39.966128 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.945574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:46125)
0619 06:54:39.946276 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:39.946314 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:39.948102 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:39.948938 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:39.948950 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:39.949804 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:54:39.950427 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.950444 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.951595 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.951602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.953623 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:39.953636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.954576 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.954584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:39.954927 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.955716 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:54:39.955751 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:54:39.958285 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:54:39.961828 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.961837 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.961856 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.962303 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.962672 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:54:39.962677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:54:39.962681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:54:39.962886 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:54:39.963423 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:54:39.963431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:54:39.963804 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 06:54:39.964075 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 06:54:39.964642 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":275,"threads_started":1}
I20260619 06:54:39.967411 2646 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35971
I20260619 06:54:39.967743 2646 heartbeater.cc:464] Registering TS with master...
I20260619 06:54:39.968461 2646 heartbeater.cc:511] Master 127.18.39.126:35971 requested a full tablet report, sending...
I20260619 06:54:39.969882 2197 ts_manager.cc:194] Registered new tserver with Master: 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871)
I20260619 06:54:39.970800 2197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:46125
I20260619 06:54:39.980607 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:54:39.987421 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.981686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.18.39.126:35971)
0619 06:54:39.981901 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.981905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.982385 (+ 480us) server_negotiation.cc:407] Connection header received
0619 06:54:39.982655 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.982659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.982818 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.982960 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:54:39.983420 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.984123 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.985664 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.986125 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.986225 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.986659 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:54:39.986906 (+ 247us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 06:54:39.987194 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48598: token expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 06:54:39.987524 2650 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48598: token expired
W20260619 06:54:39.988060 2120 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.39.126:35971)
I20260619 06:54:40.002180 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:39.988657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.18.39.126:35971)
0619 06:54:39.988848 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:39.988853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:39.988867 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:54:39.988903 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:39.988906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:39.988999 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:39.989073 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:39.990670 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.991521 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.993109 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:39.993594 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:39.993698 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:39.996451 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:39.996456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:39.996459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:39.996496 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:39.999248 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.000001 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.000006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.000008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.000141 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.000547 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.000552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.000555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.000921 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:40.001113 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.001627 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.001815 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 06:54:40.006067 2197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48600:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:54:40.013438 2197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:54:40.041967 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.027422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51292)
0619 06:54:40.028765 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.028806 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:40.028998 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:40.030225 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:40.030233 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:40.030267 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:40.030947 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.030967 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.033519 (+ 2552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.033527 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.037275 (+ 3748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.037307 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.040438 (+ 3131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.040448 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.040909 (+ 461us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.041044 (+ 135us) client_negotiation.cc:769] Sending connection context
0619 06:54:40.041211 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 06:54:40.041485 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"mutex_wait_us":223,"spinlock_wait_cycles":5888,"thread_start_us":1608,"threads_started":1}
I20260619 06:54:40.042168 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.027335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.18.39.67:44871)
0619 06:54:40.028088 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.028096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.028916 (+ 820us) server_negotiation.cc:407] Connection header received
0619 06:54:40.029247 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.029258 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.029547 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.029733 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:40.031240 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.033408 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.037555 (+ 4147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.039714 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.039860 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.040191 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.041413 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.041681 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":427,"threads_started":1}
I20260619 06:54:40.042788 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.027717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.18.39.65:42809)
0619 06:54:40.028322 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.028331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.028998 (+ 667us) server_negotiation.cc:407] Connection header received
0619 06:54:40.029259 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.029267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.029698 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.029914 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:40.031213 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.033274 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.037581 (+ 4307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.039706 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.039897 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.040191 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.042257 (+ 2066us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.042455 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":311,"threads_started":1}
I20260619 06:54:40.044699 2658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.027052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44871 (local address 127.0.0.1:51106)
0619 06:54:40.028765 (+ 1713us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.028803 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:40.028998 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:40.030158 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:40.030168 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:40.030209 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:40.030985 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.031007 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.033675 (+ 2668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.033687 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.037289 (+ 3602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.037312 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.040491 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.040500 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.040897 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.041050 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 06:54:40.042579 (+ 1529us) client_negotiation.cc:240] Negotiation successful
0619 06:54:40.042813 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1519,"mutex_wait_us":209,"spinlock_wait_cycles":3328,"thread_start_us":1345,"threads_started":1}
I20260619 06:54:40.047230 2318 tablet_service.cc:1467] Processing CreateTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 (DEFAULT_TABLE table=test-table [id=0a6f67704b594d80b3bd197dd3210afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:40.048115 2581 tablet_service.cc:1467] Processing CreateTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 (DEFAULT_TABLE table=test-table [id=0a6f67704b594d80b3bd197dd3210afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:40.048640 2318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868ad091e3054f9ebd189668cd9e7cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:40.049324 2581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868ad091e3054f9ebd189668cd9e7cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:40.058069 2383 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:40.061800 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.029317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36667 (local address 127.0.0.1:33702)
0619 06:54:40.041563 (+ 12246us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.041584 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:40.041724 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:40.045978 (+ 4254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:40.045985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:40.046018 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:40.046915 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.046929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.055762 (+ 8833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.055769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.057901 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.057924 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.060943 (+ 3019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.060950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.061132 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.061171 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:54:40.061236 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 06:54:40.061409 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12090,"thread_start_us":12488,"threads_started":1}
I20260619 06:54:40.062287 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.029424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.18.39.66:36667)
0619 06:54:40.045035 (+ 15611us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.045047 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.045076 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:54:40.045183 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.045191 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.045474 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.045682 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:40.053879 (+ 8197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.055574 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.059150 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.060654 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.061487 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.061700 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.061786 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.061956 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15417,"thread_start_us":23978,"threads_started":1}
I20260619 06:54:40.063448 2663 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Bootstrap starting.
I20260619 06:54:40.064261 2662 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Bootstrap starting.
I20260619 06:54:40.066388 2450 tablet_service.cc:1467] Processing CreateTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 (DEFAULT_TABLE table=test-table [id=0a6f67704b594d80b3bd197dd3210afe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:54:40.067448 2663 tablet_bootstrap.cc:654] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:40.067679 2662 tablet_bootstrap.cc:654] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:40.067847 2450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 868ad091e3054f9ebd189668cd9e7cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:54:40.068666 2662 log.cc:824] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:40.068706 2663 log.cc:824] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:40.071703 2662 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: No bootstrap required, opened a new log
I20260619 06:54:40.071995 2662 ts_tablet_manager.cc:1397] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:54:40.072517 2663 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: No bootstrap required, opened a new log
I20260619 06:54:40.072783 2663 ts_tablet_manager.cc:1397] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 06:54:40.079487 2662 raft_consensus.cc:348] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.079916 2662 raft_consensus.cc:374] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:40.080027 2662 raft_consensus.cc:729] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6ecc1898854cabb58386b79ecfe12f, State: Initialized, Role: FOLLOWER
I20260619 06:54:40.080668 2662 consensus_queue.cc:260] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.082511 2662 ts_tablet_manager.cc:1428] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent starting tablet: real 0.010s user 0.002s sys 0.008s
I20260619 06:54:40.084425 2663 raft_consensus.cc:348] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.085031 2663 raft_consensus.cc:374] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:40.085151 2663 raft_consensus.cc:729] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a8c7e1341d46f2a75f8a639839bbe6, State: Initialized, Role: FOLLOWER
I20260619 06:54:40.085949 2663 consensus_queue.cc:260] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.086440 2667 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Bootstrap starting.
I20260619 06:54:40.088155 2646 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:40.088840 2663 ts_tablet_manager.cc:1428] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260619 06:54:40.090289 2667 tablet_bootstrap.cc:654] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Neither blocks nor log segments found. Creating new log.
I20260619 06:54:40.091549 2667 log.cc:824] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:54:40.094290 2667 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: No bootstrap required, opened a new log
I20260619 06:54:40.094563 2667 ts_tablet_manager.cc:1397] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 06:54:40.101986 2667 raft_consensus.cc:348] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.102389 2667 raft_consensus.cc:374] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:54:40.102491 2667 raft_consensus.cc:729] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a5b9d6b429b4b75bddcd767ccff8af5, State: Initialized, Role: FOLLOWER
I20260619 06:54:40.103080 2667 consensus_queue.cc:260] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.104141 2515 heartbeater.cc:503] Master 127.18.39.126:35971 was elected leader, sending a full tablet report...
I20260619 06:54:40.104809 2667 ts_tablet_manager.cc:1428] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260619 06:54:40.139479 2668 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:54:40.139830 2668 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.141621 2668 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
I20260619 06:54:40.148283 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.142118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:43887 (local address 127.18.39.65:42809)
0619 06:54:40.142346 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.142354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.142660 (+ 306us) server_negotiation.cc:407] Connection header received
0619 06:54:40.142886 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.142891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.142952 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.143060 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:40.143542 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.144500 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.146485 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.147252 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.147426 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.147500 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.147876 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.148004 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:54:40.148298 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.142117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.18.39.67:43887)
0619 06:54:40.142532 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.142557 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:40.142726 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:40.143158 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:40.143164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:40.143194 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:40.143381 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.143395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.144623 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.144631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.146332 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.146344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.147410 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.147416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.147655 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.147736 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 06:54:40.147839 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:54:40.147974 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":109}
I20260619 06:54:40.148573 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.142108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:38617 (local address 127.18.39.66:36667)
0619 06:54:40.142327 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.142333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.143061 (+ 728us) server_negotiation.cc:407] Connection header received
0619 06:54:40.143257 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.143261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.143319 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.143391 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:54:40.143947 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.144866 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.146683 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.147440 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.147633 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.147713 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.148009 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.148140 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:54:40.148656 2671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.142163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36667 (local address 127.18.39.67:38617)
0619 06:54:40.142957 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.142970 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 06:54:40.143106 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:54:40.143431 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:54:40.143438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:54:40.143468 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:54:40.143781 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.143789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.145062 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.145066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.146526 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:54:40.146538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.147663 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.147669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:54:40.147853 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.147875 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:54:40.147978 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:54:40.148100 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":7,"thread_start_us":439,"threads_started":1}
I20260619 06:54:40.149457 2338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868ad091e3054f9ebd189668cd9e7cc7" candidate_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6ecc1898854cabb58386b79ecfe12f" is_pre_election: true
I20260619 06:54:40.149580 2470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868ad091e3054f9ebd189668cd9e7cc7" candidate_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" is_pre_election: true
I20260619 06:54:40.150098 2470 raft_consensus.cc:2454] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a8c7e1341d46f2a75f8a639839bbe6 in term 0.
I20260619 06:54:40.150110 2338 raft_consensus.cc:2454] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a8c7e1341d46f2a75f8a639839bbe6 in term 0.
I20260619 06:54:40.150828 2537 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5; no voters:
I20260619 06:54:40.151243 2668 raft_consensus.cc:2781] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:54:40.151378 2668 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:54:40.151479 2668 raft_consensus.cc:3037] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:40.153802 2668 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.154482 2668 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [CANDIDATE]: Term 1 election: Requested vote from peers 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
I20260619 06:54:40.154963 2338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868ad091e3054f9ebd189668cd9e7cc7" candidate_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6ecc1898854cabb58386b79ecfe12f"
I20260619 06:54:40.155040 2470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "868ad091e3054f9ebd189668cd9e7cc7" candidate_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
I20260619 06:54:40.155191 2338 raft_consensus.cc:3037] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:40.155279 2470 raft_consensus.cc:3037] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:54:40.157609 2338 raft_consensus.cc:2454] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a8c7e1341d46f2a75f8a639839bbe6 in term 1.
I20260619 06:54:40.157702 2470 raft_consensus.cc:2454] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a8c7e1341d46f2a75f8a639839bbe6 in term 1.
I20260619 06:54:40.158110 2534 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a8c7e1341d46f2a75f8a639839bbe6, 0c6ecc1898854cabb58386b79ecfe12f; no voters:
I20260619 06:54:40.158463 2668 raft_consensus.cc:2781] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:54:40.158949 2668 raft_consensus.cc:686] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 LEADER]: Becoming Leader. State: Replica: 05a8c7e1341d46f2a75f8a639839bbe6, State: Running, Role: LEADER
I20260619 06:54:40.159513 2668 consensus_queue.cc:237] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:54:40.164132 2198 catalog_manager.cc:5495] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } health_report { overall_health: UNKNOWN } } }
I20260619 06:54:40.181613 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.175915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.18.39.67:44871)
0619 06:54:40.176147 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.176155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.176268 (+ 113us) server_negotiation.cc:407] Connection header received
0619 06:54:40.176581 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.176585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.176667 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.176846 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:54:40.177216 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.178019 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.179399 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.179851 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.179992 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.180518 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:54:40.181025 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:54:40.181184 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.181241 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.181325 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:54:40.190275 2470 raft_consensus.cc:1261] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Refusing update from remote peer 05a8c7e1341d46f2a75f8a639839bbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:54:40.190315 2338 raft_consensus.cc:1261] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Refusing update from remote peer 05a8c7e1341d46f2a75f8a639839bbe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:54:40.191308 2672 consensus_queue.cc:1035] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:54:40.191653 2668 consensus_queue.cc:1035] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:54:40.193944 2647 tablet.cc:2369] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:54:40.208142 2677 mvcc.cc:205] Tried to move back new op lower bound from 7298466120447533056 to 7298466120337506304. Current Snapshot: MvccSnapshot[applied={T|T < 7298466120447533056}]
I20260619 06:54:40.222713 2197 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:54:40.223027 2197 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48600:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:54:40.226842 2197 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Sending DeleteTablet for 3 replicas of tablet 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:54:40.228193 2581 tablet_service.cc:1514] Processing DeleteTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:51106
I20260619 06:54:40.228199 2318 tablet_service.cc:1514] Processing DeleteTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:51292
I20260619 06:54:40.228444 2450 tablet_service.cc:1514] Processing DeleteTablet for tablet 868ad091e3054f9ebd189668cd9e7cc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:54:40 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:33702
I20260619 06:54:40.229628 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2235
I20260619 06:54:40.245523 2686 tablet_replica.cc:330] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: stopping tablet replica
I20260619 06:54:40.246176 2685 tablet_replica.cc:330] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: stopping tablet replica
I20260619 06:54:40.246546 2687 tablet_replica.cc:330] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: stopping tablet replica
I20260619 06:54:40.247015 2687 raft_consensus.cc:2229] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:54:40.247433 2685 raft_consensus.cc:2229] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:54:40.247570 2687 raft_consensus.cc:2258] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:40.247108 2686 raft_consensus.cc:2229] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:54:40.249245 2686 raft_consensus.cc:2258] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:40.249523 2685 raft_consensus.cc:2258] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:54:40.249899 2687 ts_tablet_manager.cc:1905] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:40.251875 2685 ts_tablet_manager.cc:1905] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:40.254105 2686 ts_tablet_manager.cc:1905] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:54:40.257318 2687 ts_tablet_manager.cc:1918] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:54:40.257526 2687 log.cc:1196] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:54:40.258093 2687 ts_tablet_manager.cc:1939] T 868ad091e3054f9ebd189668cd9e7cc7 P 8a5b9d6b429b4b75bddcd767ccff8af5: Deleting consensus metadata
I20260619 06:54:40.259867 2686 ts_tablet_manager.cc:1918] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:54:40.260075 2686 log.cc:1196] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:54:40.260049 2185 catalog_manager.cc:4841] TS 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): tablet 868ad091e3054f9ebd189668cd9e7cc7 (table test-table [id=0a6f67704b594d80b3bd197dd3210afe]) successfully deleted
I20260619 06:54:40.260563 2686 ts_tablet_manager.cc:1939] T 868ad091e3054f9ebd189668cd9e7cc7 P 05a8c7e1341d46f2a75f8a639839bbe6: Deleting consensus metadata
I20260619 06:54:40.253636 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.230881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.18.39.65:42809)
0619 06:54:40.231061 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.231067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.231088 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:54:40.231147 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.231153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.231265 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.231369 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:40.233617 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.234790 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.237377 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.238057 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.238201 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.244194 (+ 5993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:40.244222 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:40.244261 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:40.244327 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:40.249098 (+ 4771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.250159 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.250169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.250173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.250349 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.250849 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.250853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.250856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.251340 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:40.251649 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.252984 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.253224 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:54:40.262374 2378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:40.262755 2185 catalog_manager.cc:4841] TS 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): tablet 868ad091e3054f9ebd189668cd9e7cc7 (table test-table [id=0a6f67704b594d80b3bd197dd3210afe]) successfully deleted
I20260619 06:54:40.353219 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2235
I20260619 06:54:40.382476 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2387
W20260619 06:54:40.382658 2182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.39.65:42809 (error 108)
I20260619 06:54:40.384925 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.384523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51310)
0619 06:54:40.384680 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.384776 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:54:40.385233 2182 catalog_manager.cc:4584] TS 0c6ecc1898854cabb58386b79ecfe12f (127.18.39.65:42809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 868ad091e3054f9ebd189668cd9e7cc7: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
I20260619 06:54:40.404422 2660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.383280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.18.39.66:36667)
0619 06:54:40.383468 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.383475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.383785 (+ 310us) server_negotiation.cc:407] Connection header received
0619 06:54:40.383864 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.383869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.383962 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.384076 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:40.385940 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.386670 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.388612 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.389079 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.389175 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.397848 (+ 8673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:40.397866 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:40.397881 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:40.397939 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:40.400945 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.401885 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.401893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.401895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.402025 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.402422 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.402425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.402427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.402996 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:40.403297 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.403821 (+ 524us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.404044 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:54:40.405159 2510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:40.439199 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.438605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51312)
0619 06:54:40.438859 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.438987 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 06:54:40.440440 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.439975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51320)
0619 06:54:40.440203 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.440283 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260619 06:54:40.496982 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2387
I20260619 06:54:40.501051 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.500424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51336)
0619 06:54:40.500706 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.500858 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 06:54:40.502300 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.501855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51350)
0619 06:54:40.502062 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.502162 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 06:54:40.515208 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2518
I20260619 06:54:40.532951 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.516114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.18.39.67:44871)
0619 06:54:40.516320 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:54:40.516326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:54:40.516372 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:54:40.516458 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:54:40.516464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:54:40.516590 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:54:40.516696 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:54:40.518066 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.519337 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.521385 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:54:40.522268 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:54:40.522418 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:54:40.526204 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:54:40.526218 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:54:40.526238 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:54:40.526288 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:54:40.529541 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.530569 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.530581 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.530586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.530805 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:54:40.531373 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:54:40.531377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:54:40.531380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:54:40.531792 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:54:40.532204 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0619 06:54:40.532425 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 06:54:40.532621 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:54:40.533857 2641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:40.583458 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.582981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51366)
0619 06:54:40.583189 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.583296 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260619 06:54:40.584515 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.584132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51382)
0619 06:54:40.584332 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.584400 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260619 06:54:40.658849 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2518
I20260619 06:54:40.679459 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2165
I20260619 06:54:40.680621 2228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:54:40.796885 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:54:40.795489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:42809 (local address 127.0.0.1:51388)
0619 06:54:40.796431 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0619 06:54:40.796558 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:42809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":796}
I20260619 06:54:40.797237 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2165
I20260619 06:55:00.819597 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--webserver_interface=127.18.39.126
--webserver_port=43419
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:01.175349 2693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:01.175666 2693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:01.175760 2693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:01.183383 2693 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:55:01.183497 2693 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:55:01.183537 2693 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:55:01.183588 2693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:01.183653 2693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:01.183723 2693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:01.183768 2693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:01.183823 2693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:01.194007 2693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=43419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:01.195329 2693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:01.196727 2693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:01.209118 2698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:01.209081 2701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:01.209160 2699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:01.209836 2693 server_base.cc:1034] running on GCE node
I20260619 06:55:01.210623 2693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:01.212049 2693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:01.213289 2693 hybrid_clock.cc:648] HybridClock initialized: now 1781852101213212 us; error 62 us; skew 500 ppm
I20260619 06:55:01.213690 2693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:01 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852101, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:01.220228 2693 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:55:01.221930 2693 webserver.cc:460] Webserver started at http://127.18.39.126:43419/ using document root <none> and password file <none>
I20260619 06:55:01.222684 2693 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:01.222817 2693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:01.228437 2693 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:55:01.231442 2708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:01.232771 2693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:01.232957 2693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "076f444fe6339b01a57611f055282507"
server_key_iv: "9205c24885c60ebf4530a7d10abe59be"
server_key_version: "encryptionkey@0"
I20260619 06:55:01.234004 2693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:01.250841 2693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:01.255599 2693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:01.255960 2693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:01.274900 2693 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35971
I20260619 06:55:01.274932 2759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35971 every 8 connection(s)
I20260619 06:55:01.276489 2693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:01.278772 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2693
I20260619 06:55:01.279610 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:42809
--local_ip_for_outbound_sockets=127.18.39.65
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=36011
--webserver_interface=127.18.39.65
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:55:01.283762 2760 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:55:01.287289 2760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap starting.
I20260619 06:55:01.318334 2760 log.cc:824] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:01.339721 2760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:55:01.340428 2760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Bootstrap complete.
I20260619 06:55:01.355669 2760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:55:01.357010 2760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Initialized, Role: FOLLOWER
I20260619 06:55:01.357857 2760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:55:01.358121 2760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:01.358245 2760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:01.358438 2760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 2 FOLLOWER]: Advancing to term 3
I20260619 06:55:01.362848 2760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:55:01.363386 2760 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43ddb2ef8b4076bf0e2fac99b78d27; no voters:
I20260619 06:55:01.364055 2760 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 06:55:01.364264 2764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 06:55:01.365449 2764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [term 3 LEADER]: Becoming Leader. State: Replica: 0c43ddb2ef8b4076bf0e2fac99b78d27, State: Running, Role: LEADER
I20260619 06:55:01.366034 2764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } }
I20260619 06:55:01.366204 2760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:01.369014 2766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43ddb2ef8b4076bf0e2fac99b78d27. Latest consensus state: current_term: 3 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:55:01.369475 2766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:01.370149 2765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43ddb2ef8b4076bf0e2fac99b78d27" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35971 } } }
I20260619 06:55:01.370500 2765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:01.379063 2771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:01.389043 2771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:01.390664 2771 catalog_manager.cc:1259] Loaded cluster ID: 9a72e37dd8d74150b40bc15040ab8e86
I20260619 06:55:01.390776 2771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:01.395584 2771 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:01.397269 2771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Loaded TSK: 0
I20260619 06:55:01.397454 2771 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c43ddb2ef8b4076bf0e2fac99b78d27: Loaded TSK: 1
F20260619 06:55:01.398559 2771 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f56eba269ad google::LogMessage::Fail() at ??:0
@ 0x7f56eba2988a google::LogMessage::SendToLog() at ??:0
@ 0x7f56eba264de google::LogMessage::Flush() at ??:0
@ 0x7f56eba27a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f56ef78a7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f56ef77973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f57014551c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5701454783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f57014e02a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f57014e00c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f570152a5e9 std::function<>::operator()() at ??:0
@ 0x7f5701458cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5701456c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f57014de610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f57014de439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f56ee672345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f56ee67591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f56ee675799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f56ee650eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f56f192a6db start_thread at ??:0
@ 0x7f56ea8c061f clone at ??:0
W20260619 06:55:01.707274 2762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:01.707569 2762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:01.707728 2762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:01.715389 2762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:01.715515 2762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:01.715689 2762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:01.725958 2762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:42809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=36011
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:01.727370 2762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:01.728783 2762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:01.742648 2807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:01.742684 2810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:01.742599 2808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:01.743508 2762 server_base.cc:1034] running on GCE node
I20260619 06:55:01.744192 2762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:01.745196 2762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:01.746429 2762 hybrid_clock.cc:648] HybridClock initialized: now 1781852101746372 us; error 53 us; skew 500 ppm
I20260619 06:55:01.746757 2762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:01 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852101, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:01.750102 2762 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:55:01.751684 2762 webserver.cc:460] Webserver started at http://127.18.39.65:36011/ using document root <none> and password file <none>
I20260619 06:55:01.752419 2762 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:01.752570 2762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:01.758626 2762 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:55:01.761829 2817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:01.763410 2762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:55:01.763622 2762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "0c6ecc1898854cabb58386b79ecfe12f"
format_stamp: "Formatted at 2026-06-19 06:54:15 on dist-test-slave-1h5s"
server_key: "34f7fdf6064668eae1a8a205115ebb26"
server_key_iv: "f9be742e2bedd774bfa715fb1a3549e4"
server_key_version: "encryptionkey@0"
I20260619 06:55:01.764663 2762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:01.795213 2762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:01.799751 2762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:01.800055 2762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:01.801080 2762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:01.803076 2824 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260619 06:55:01.808940 2762 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 06:55:01.809147 2762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260619 06:55:01.809307 2762 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 06:55:01.812973 2762 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 06:55:01.813118 2762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20260619 06:55:01.813496 2824 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Bootstrap starting.
I20260619 06:55:01.821801 2824 log.cc:824] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:01.837107 2824 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:55:01.838047 2824 tablet_bootstrap.cc:492] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Bootstrap complete.
I20260619 06:55:01.838726 2824 ts_tablet_manager.cc:1397] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent bootstrapping tablet: real 0.025s user 0.024s sys 0.000s
I20260619 06:55:01.849480 2824 raft_consensus.cc:348] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:01.851248 2824 raft_consensus.cc:729] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6ecc1898854cabb58386b79ecfe12f, State: Initialized, Role: FOLLOWER
I20260619 06:55:01.852233 2824 consensus_queue.cc:260] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:01.854416 2824 ts_tablet_manager.cc:1428] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260619 06:55:01.859229 2762 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:42809
I20260619 06:55:01.859270 2932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:42809 every 8 connection(s)
I20260619 06:55:01.860569 2762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:01.865681 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:01.863622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:51747)
0619 06:55:01.864405 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:01.864663 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":314,"threads_started":1}
W20260619 06:55:01.866341 2933 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:35971 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:35971: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
I20260619 06:55:01.867213 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:01.866906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:57273)
0619 06:55:01.867043 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:01.867111 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 06:55:01.868093 2935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:01.867744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:34833)
0619 06:55:01.867893 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:01.867970 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:55:01.868348 2933 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:55:01.870268 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2762
I20260619 06:55:01.870921 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:36667
--local_ip_for_outbound_sockets=127.18.39.66
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=36931
--webserver_interface=127.18.39.66
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:02.269793 2936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:02.270252 2936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:02.270373 2936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:02.279188 2936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:02.279362 2936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:02.279505 2936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:02.291333 2936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:36667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=36931
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:02.293119 2936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:02.295153 2936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:02.312110 2942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:02.311929 2941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:02.312079 2944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:02.313262 2936 server_base.cc:1034] running on GCE node
I20260619 06:55:02.314134 2936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:02.315368 2936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:02.316606 2936 hybrid_clock.cc:648] HybridClock initialized: now 1781852102316519 us; error 67 us; skew 500 ppm
I20260619 06:55:02.317049 2936 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:02 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852102, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:02.320937 2936 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:55:02.323140 2936 webserver.cc:460] Webserver started at http://127.18.39.66:36931/ using document root <none> and password file <none>
I20260619 06:55:02.324077 2936 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:02.324271 2936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:02.331629 2936 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:55:02.335175 2951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.336918 2936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:55:02.337152 2936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "8a5b9d6b429b4b75bddcd767ccff8af5"
format_stamp: "Formatted at 2026-06-19 06:54:16 on dist-test-slave-1h5s"
server_key: "3f75d99f6570444cb936afa126a95e2a"
server_key_iv: "8492a1f73730b107af954c88af4555f2"
server_key_version: "encryptionkey@0"
I20260619 06:55:02.338411 2936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:02.360212 2936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:02.366532 2936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:02.366861 2936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:02.368193 2936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:02.369896 2936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:02.370000 2936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.370177 2936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:02.370266 2936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.407658 2936 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:36667
I20260619 06:55:02.407711 3063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:36667 every 8 connection(s)
I20260619 06:55:02.409587 2936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:02.415668 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.413394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:47881)
0619 06:55:02.414295 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.414584 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":333,"threads_started":1}
W20260619 06:55:02.416585 3064 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:35971 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:35971: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
I20260619 06:55:02.417680 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.417273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:41961)
0619 06:55:02.417479 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.417558 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 06:55:02.418553 3066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.418228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:44981)
0619 06:55:02.418368 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.418439 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:55:02.418816 3064 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:55:02.419129 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 2936
I20260619 06:55:02.419809 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:44871
--local_ip_for_outbound_sockets=127.18.39.67
--tserver_master_addrs=127.18.39.126:35971
--webserver_port=38947
--webserver_interface=127.18.39.67
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:02.820269 3067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:02.820627 3067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:02.820752 3067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:02.828802 3067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:02.828960 3067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:02.829100 3067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:02.840559 3067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:44871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=38947
--tserver_master_addrs=127.18.39.126:35971
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:02.842586 3067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:02.844453 3067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:02.860095 3073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:02.860661 3072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:02.866413 3075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:02.866750 3067 server_base.cc:1034] running on GCE node
I20260619 06:55:02.867561 3067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:02.869004 3067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:02.870143 3078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.869076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:43831)
0619 06:55:02.869709 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.869818 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":305,"threads_started":1}
I20260619 06:55:02.870362 3067 hybrid_clock.cc:648] HybridClock initialized: now 1781852102870251 us; error 79 us; skew 500 ppm
I20260619 06:55:02.870757 3067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:02 dist-test-slave-1h5s krb5kdc[1613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852102, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:02.874364 3067 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:55:02.876091 3067 webserver.cc:460] Webserver started at http://127.18.39.67:38947/ using document root <none> and password file <none>
I20260619 06:55:02.877003 3067 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:02.877166 3067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:02.883883 3067 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:55:02.887332 3083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.889057 3067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:55:02.889279 3067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "05a8c7e1341d46f2a75f8a639839bbe6"
format_stamp: "Formatted at 2026-06-19 06:54:17 on dist-test-slave-1h5s"
server_key: "f0ee6779a0e47de1d8b9fdee65fa4c2d"
server_key_iv: "87c21d928b57702db97e126430600d24"
server_key_version: "encryptionkey@0"
I20260619 06:55:02.890648 3067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:02.909554 3067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:02.915115 3067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:02.915469 3067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:02.916762 3067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:02.918458 3067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:02.918593 3067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.918751 3067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:02.918846 3067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:02.956897 3067 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:44871
I20260619 06:55:02.956951 3195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:44871 every 8 connection(s)
I20260619 06:55:02.958762 3067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:02.961687 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3067
I20260619 06:55:02.966055 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.962732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:37675)
0619 06:55:02.964203 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.964674 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":482,"threads_started":1}
W20260619 06:55:02.966893 3196 heartbeater.cc:644] Failed to heartbeat to 127.18.39.126:35971 (0 consecutive failures): Network error: Failed to ping master at 127.18.39.126:35971: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
I20260619 06:55:02.967931 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.967550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:43737)
0619 06:55:02.967738 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.967813 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 06:55:02.968900 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:02.968567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:48155)
0619 06:55:02.968705 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:02.968789 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 06:55:02.969158 3196 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:55:03.420279 3200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.419425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:33669)
0619 06:55:03.419915 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:03.420034 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":230,"threads_started":1}
I20260619 06:55:03.527834 3201 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:03.528280 3201 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:03.530251 3201 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
Jun 19 06:55:03 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852101, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
Jun 19 06:55:03 dist-test-slave-1h5s krb5kdc[1613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852101, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:55:03.557891 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.531884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:55223 (local address 127.18.39.66:36667)
0619 06:55:03.532519 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:03.532533 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:03.532573 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:55:03.534770 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:03.534780 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:03.535245 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:03.535645 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:03.538684 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.540650 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.543636 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.544675 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.544870 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:03.550070 (+ 5200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:03.550087 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:03.550105 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:03.550149 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:03.553658 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:03.554684 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:03.554693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:03.554700 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:03.554869 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:03.555491 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:03.555499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:03.555504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:03.556053 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:03.556261 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:03.557038 (+ 777us) server_negotiation.cc:299] Negotiation successful
0619 06:55:03.557464 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":255,"threads_started":1}
I20260619 06:55:03.557904 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.530919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:58149 (local address 127.18.39.67:44871)
0619 06:55:03.531475 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:03.531493 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:03.531544 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:55:03.534672 (+ 3128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:03.534685 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:03.535246 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:03.535642 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:03.538684 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.540650 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.543627 (+ 2977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.544675 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.544870 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:03.550164 (+ 5294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:03.550171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:03.550209 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:03.550267 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:03.553641 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:03.554805 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:03.554812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:03.554817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:03.555065 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:03.555526 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:03.555530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:03.555532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:03.556054 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:03.556260 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:03.557027 (+ 767us) server_negotiation.cc:299] Negotiation successful
0619 06:55:03.557456 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":283,"threads_started":1}
I20260619 06:55:03.557991 3202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.530688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:44871 (local address 127.18.39.65:58149)
0619 06:55:03.531262 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:03.531311 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:03.531962 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:03.535846 (+ 3884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:03.535857 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:03.537149 (+ 1292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:03.538414 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:03.538448 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.540873 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.540877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:03.543430 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:03.543447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.544955 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.544963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:03.545398 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:03.546348 (+ 950us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:03.546391 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:03.549912 (+ 3521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:03.553796 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:03.553808 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:03.553826 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:03.554587 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:03.555188 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:03.555192 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:03.555194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:03.555407 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:03.556195 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:03.556207 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:03.556540 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 06:55:03.556811 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 06:55:03.557455 (+ 644us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":499,"spinlock_wait_cycles":53888,"thread_start_us":299,"threads_started":1}
I20260619 06:55:03.558211 3204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.531451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:36667 (local address 127.18.39.65:55223)
0619 06:55:03.532067 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:03.532092 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:03.532227 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:03.535846 (+ 3619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:03.535857 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:03.537149 (+ 1292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:03.538414 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:03.538447 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.540871 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.540877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:03.543430 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:03.543447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:03.544955 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:03.544963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:03.545400 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:03.546348 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:03.546391 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:03.549702 (+ 3311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:03.553827 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:03.553839 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:03.553842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:03.554521 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:03.555006 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:03.555011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:03.555014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:03.555321 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:03.556195 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:03.556204 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:03.556540 (+ 336us) client_negotiation.cc:769] Sending connection context
0619 06:55:03.556813 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 06:55:03.557614 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":755,"spinlock_wait_cycles":91264,"thread_start_us":260,"threads_started":1}
W20260619 06:55:03.561427 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
W20260619 06:55:03.561810 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:55:03.561957 2821 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f; no voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5
I20260619 06:55:03.562398 3201 raft_consensus.cc:2726] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:55:03.871577 3202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.871141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:52485)
0619 06:55:03.871329 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:03.871422 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 06:55:03.970721 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:03.969808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:49855)
0619 06:55:03.970315 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:03.970450 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":250,"threads_started":1}
I20260619 06:55:04.422092 3207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:04.421304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:39621)
0619 06:55:04.421827 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:04.421914 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":257,"threads_started":1}
I20260619 06:55:04.873317 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:04.872540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:45817)
0619 06:55:04.873038 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:04.873127 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":265,"threads_started":1}
I20260619 06:55:04.972352 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:04.971686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:45331)
0619 06:55:04.972115 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:04.972194 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260619 06:55:05.247056 3210 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:05.247294 3210 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:05.248021 3210 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
W20260619 06:55:05.249122 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
W20260619 06:55:05.249382 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:55:05.249476 2821 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f; no voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5
I20260619 06:55:05.249781 3210 raft_consensus.cc:2726] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:55:05.424010 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:05.423147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:49967)
0619 06:55:05.423655 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:05.423752 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":282,"threads_started":1}
I20260619 06:55:05.875095 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:05.874379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:46087)
0619 06:55:05.874835 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:05.874928 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":240,"threads_started":1}
I20260619 06:55:05.974244 3213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:05.973490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:38807)
0619 06:55:05.973989 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:05.974078 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260619 06:55:06.426383 3214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:06.425426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:43225)
0619 06:55:06.426031 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:06.426147 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":313,"threads_started":1}
I20260619 06:55:06.877571 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:06.876471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:51647)
0619 06:55:06.877112 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:06.877242 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":311,"threads_started":1}
I20260619 06:55:06.976362 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:06.975453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:47759)
0619 06:55:06.976040 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:06.976144 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":297,"threads_started":1}
I20260619 06:55:07.428318 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:07.427547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:52271)
0619 06:55:07.428033 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:07.428130 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":248,"threads_started":1}
I20260619 06:55:07.785707 3218 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:07.785948 3218 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:07.786767 3218 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
W20260619 06:55:07.787909 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
W20260619 06:55:07.788160 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:55:07.788285 2821 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f; no voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5
I20260619 06:55:07.788592 3218 raft_consensus.cc:2726] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:55:07.879396 3219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:07.878541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:54113)
0619 06:55:07.879074 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:07.879188 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":259,"threads_started":1}
I20260619 06:55:07.978488 3220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:07.977672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:37539)
0619 06:55:07.978159 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:07.978253 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
I20260619 06:55:08.430066 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:08.429257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:41595)
0619 06:55:08.429782 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:08.429867 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":309,"threads_started":1}
I20260619 06:55:08.880980 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:08.880252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:57645)
0619 06:55:08.880679 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:08.880768 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":227,"threads_started":1}
I20260619 06:55:08.980257 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:08.979478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:39733)
0619 06:55:08.980006 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:08.980084 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":253,"threads_started":1}
I20260619 06:55:09.432548 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:09.431406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:43561)
0619 06:55:09.432176 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:09.432285 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":376,"threads_started":1}
I20260619 06:55:09.882803 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:09.881966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:47073)
0619 06:55:09.882492 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:09.882598 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":283,"threads_started":1}
I20260619 06:55:09.982416 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:09.981448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:37209)
0619 06:55:09.982090 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:09.982189 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":300,"threads_started":1}
I20260619 06:55:10.434558 3227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:10.433739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:35455)
0619 06:55:10.434236 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:10.434350 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
I20260619 06:55:10.884526 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:10.883767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:40571)
0619 06:55:10.884240 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:10.884332 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260619 06:55:10.984282 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:10.983555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:46437)
0619 06:55:10.983984 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:10.984073 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":225,"threads_started":1}
I20260619 06:55:11.436379 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:11.435559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:53763)
0619 06:55:11.436110 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:11.436210 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":266,"threads_started":1}
I20260619 06:55:11.886394 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:11.885659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:42867)
0619 06:55:11.886124 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:11.886209 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":241,"threads_started":1}
I20260619 06:55:11.986261 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:11.985491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:47785)
0619 06:55:11.986014 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:11.986107 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":239,"threads_started":1}
I20260619 06:55:12.438249 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:12.437508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:36413)
0619 06:55:12.437975 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:12.438071 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":224,"threads_started":1}
I20260619 06:55:12.598380 3234 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:12.598603 3234 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:12.599264 3234 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
W20260619 06:55:12.600210 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
W20260619 06:55:12.600442 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:55:12.600523 2821 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f; no voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5
I20260619 06:55:12.600817 3234 raft_consensus.cc:2726] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:55:12.888402 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:12.887568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:52477)
0619 06:55:12.888104 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:12.888202 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":246,"threads_started":1}
I20260619 06:55:12.988112 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:12.987356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:36477)
0619 06:55:12.987831 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:12.987919 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":252,"threads_started":1}
I20260619 06:55:13.440234 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:13.439406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:54561)
0619 06:55:13.439928 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:13.440027 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":263,"threads_started":1}
I20260619 06:55:13.890061 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:13.889310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:49905)
0619 06:55:13.889773 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:13.889864 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":225,"threads_started":1}
I20260619 06:55:13.989956 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:13.989148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:53611)
0619 06:55:13.989714 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:13.989801 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":311,"threads_started":1}
I20260619 06:55:14.442016 3240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:14.441241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:43633)
0619 06:55:14.441751 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:14.441846 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":287,"threads_started":1}
I20260619 06:55:14.891811 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:14.890932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:39291)
0619 06:55:14.891527 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:14.891622 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":255,"threads_started":1}
I20260619 06:55:14.991755 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:14.990976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:58171)
0619 06:55:14.991475 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:14.991585 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":277,"threads_started":1}
I20260619 06:55:15.444262 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:15.443193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:48465)
0619 06:55:15.443887 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:15.444039 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":406,"threads_started":1}
I20260619 06:55:15.893968 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:15.892904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:57817)
0619 06:55:15.893541 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:15.893638 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":315,"threads_started":1}
I20260619 06:55:15.993883 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:15.992885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:40609)
0619 06:55:15.993522 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:15.993652 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":274,"threads_started":1}
I20260619 06:55:16.446146 3246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:16.445275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:48583)
0619 06:55:16.445823 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:16.445929 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":304,"threads_started":1}
I20260619 06:55:16.885950 3247 raft_consensus.cc:482] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:16.886139 3247 raft_consensus.cc:504] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05a8c7e1341d46f2a75f8a639839bbe6" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 44871 } } peers { permanent_uuid: "0c6ecc1898854cabb58386b79ecfe12f" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 42809 } } peers { permanent_uuid: "8a5b9d6b429b4b75bddcd767ccff8af5" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 36667 } }
I20260619 06:55:16.886813 3247 leader_election.cc:288] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871), 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667)
W20260619 06:55:16.887981 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 05a8c7e1341d46f2a75f8a639839bbe6 (127.18.39.67:44871): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
W20260619 06:55:16.888227 2821 leader_election.cc:341] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8a5b9d6b429b4b75bddcd767ccff8af5 (127.18.39.66:36667): Not found: Tablet not found: 868ad091e3054f9ebd189668cd9e7cc7
I20260619 06:55:16.888357 2821 leader_election.cc:302] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c6ecc1898854cabb58386b79ecfe12f; no voters: 05a8c7e1341d46f2a75f8a639839bbe6, 8a5b9d6b429b4b75bddcd767ccff8af5
I20260619 06:55:16.888640 3247 raft_consensus.cc:2726] T 868ad091e3054f9ebd189668cd9e7cc7 P 0c6ecc1898854cabb58386b79ecfe12f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:55:16.896044 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:16.895208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:59849)
0619 06:55:16.895694 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:16.895802 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":243,"threads_started":1}
I20260619 06:55:16.995697 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:16.994809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:57873)
0619 06:55:16.995377 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:16.995480 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":263,"threads_started":1}
I20260619 06:55:17.448200 3250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:17.447338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.66:52599)
0619 06:55:17.447903 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:17.448005 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":307,"threads_started":1}
I20260619 06:55:17.897765 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:17.896945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.65:52235)
0619 06:55:17.897488 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:17.897587 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260619 06:55:17.972108 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2762
I20260619 06:55:17.988101 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:17.973501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47958 (local address 127.18.39.65:42809)
0619 06:55:17.974185 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:17.974199 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:17.974235 (+ 36us) server_negotiation.cc:407] Connection header received
0619 06:55:17.974337 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:17.974348 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:17.974669 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:17.974860 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:17.976315 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:17.977386 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:17.978683 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:17.979249 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:17.979347 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:17.982126 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:17.982141 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:17.982156 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:17.982197 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:17.985043 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:17.985900 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:17.985909 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:17.985914 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:17.986061 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:17.986447 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:17.986452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:17.986457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:17.986830 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:17.987163 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:17.987467 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 06:55:17.987765 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":284,"threads_started":1}
I20260619 06:55:17.989373 2928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:17.997802 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:17.996897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35971 (local address 127.18.39.67:40223)
0619 06:55:17.997477 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:17.997645 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.126:35971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":264,"threads_started":1}
I20260619 06:55:18.082607 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2762
I20260619 06:55:18.102666 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 2936
I20260619 06:55:18.116492 3255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:18.103445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.18.39.66:36667)
0619 06:55:18.103996 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:18.104001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:18.104017 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:55:18.104074 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:18.104077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:18.104137 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:18.104238 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:18.105797 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:18.106588 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:18.107951 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:18.108379 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:18.108470 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:18.111341 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:18.111347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:18.111350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:18.111381 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:18.114063 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:18.114709 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:18.114714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:18.114716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:18.114857 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:18.115178 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:18.115181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:18.115183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:18.115443 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:18.115719 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:18.115923 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 06:55:18.116088 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":333,"threads_started":1}
I20260619 06:55:18.117403 3059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:18.204324 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 2936
I20260619 06:55:18.221969 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3067
I20260619 06:55:18.236564 3257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:18.222969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.18.39.67:44871)
0619 06:55:18.223618 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:18.223623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:18.223642 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:55:18.223707 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:18.223711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:18.223790 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:18.223890 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:18.225280 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:18.226437 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:18.227680 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:18.228136 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:18.228239 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:18.230899 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:18.230905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:18.230909 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:18.230946 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:18.234113 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:18.234751 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:18.234755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:18.234758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:18.234895 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:18.235198 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:18.235202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:18.235204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:18.235472 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:18.235704 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:18.235924 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 06:55:18.236106 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":315,"threads_started":1}
I20260619 06:55:18.237434 3191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:18.324996 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3067
2026-06-19T06:55:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64750 ms)
[----------] 30 tests from SecurityITest (299403 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:18 dist-test-slave-1h5s krb5kdc[3261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:18 dist-test-slave-1h5s krb5kdc[3261](info): set up 2 sockets
Jun 19 06:55:18 dist-test-slave-1h5s krb5kdc[3261](info): commencing operation
krb5kdc: starting...
W20260619 06:55:19.441556 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:19 dist-test-slave-1h5s krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:55:19.693409 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35377
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:20.060125 3274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:20.060446 3274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:20.060619 3274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:20.068235 3274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:20.068362 3274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:20.068465 3274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:20.068578 3274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:20.068668 3274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:20.079051 3274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:20.080472 3274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:20.081965 3274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:20.095709 3279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:20.095790 3280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:20.095736 3274 server_base.cc:1034] running on GCE node
W20260619 06:55:20.095737 3282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:20.097069 3274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:20.098675 3274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:20.099884 3274 hybrid_clock.cc:648] HybridClock initialized: now 1781852120099828 us; error 44 us; skew 500 ppm
I20260619 06:55:20.100253 3274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:20 dist-test-slave-1h5s krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852120, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:20.103480 3274 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:55:20.105074 3274 webserver.cc:460] Webserver started at http://127.18.39.126:46593/ using document root <none> and password file <none>
I20260619 06:55:20.105839 3274 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:20.105971 3274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:20.106300 3274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:20.109436 3274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "f441d84acc464817915584640d2e3d28"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "2aadf6036ed9820398047164230286fb"
server_key_iv: "3514bf43ebb1406a0fb905264ea38af9"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.110241 3274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "f441d84acc464817915584640d2e3d28"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "2aadf6036ed9820398047164230286fb"
server_key_iv: "3514bf43ebb1406a0fb905264ea38af9"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.115610 3274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:20.119289 3289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:20.120646 3274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:55:20.120827 3274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "f441d84acc464817915584640d2e3d28"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "2aadf6036ed9820398047164230286fb"
server_key_iv: "3514bf43ebb1406a0fb905264ea38af9"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.121028 3274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:20.142459 3274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:20.146965 3274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:20.147282 3274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:20.166658 3274 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35377
I20260619 06:55:20.166680 3340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35377 every 8 connection(s)
I20260619 06:55:20.168274 3274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:20.168699 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3274
I20260619 06:55:20.168922 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:20.169857 18589 external_mini_cluster.cc:1428] Setting key 0087dc2944f3a829b22e5b4e0928acd1
I20260619 06:55:20.172951 3341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:55:20 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:20.186631 3341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: Bootstrap starting.
I20260619 06:55:20.189931 3341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:20.190984 3341 log.cc:824] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:20.193899 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:20.171944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.18.39.126:35377)
0619 06:55:20.172766 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:20.172777 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:20.172817 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:55:20.174575 (+ 1758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:20.174597 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:20.174978 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:20.175309 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:20.177142 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.178558 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.180532 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.181216 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.181412 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:20.186300 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:20.186309 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:20.186325 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:20.186365 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:20.189609 (+ 3244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:20.190416 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:20.190425 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:20.190431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:20.190599 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:20.190998 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:20.191003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:20.191008 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:20.191608 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:20.191964 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:20.192404 (+ 440us) server_negotiation.cc:299] Negotiation successful
0619 06:55:20.192898 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":386,"threads_started":1}
I20260619 06:55:20.194139 3341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: No bootstrap required, opened a new log
I20260619 06:55:20.202140 3341 raft_consensus.cc:348] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } }
I20260619 06:55:20.202569 3341 raft_consensus.cc:374] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:20.202680 3341 raft_consensus.cc:729] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f441d84acc464817915584640d2e3d28, State: Initialized, Role: FOLLOWER
I20260619 06:55:20.203248 3341 consensus_queue.cc:260] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } }
I20260619 06:55:20.203491 3341 raft_consensus.cc:388] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:20.203639 3341 raft_consensus.cc:482] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:20.203814 3341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:20.206386 3341 raft_consensus.cc:504] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } }
I20260619 06:55:20.206892 3341 leader_election.cc:302] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f441d84acc464817915584640d2e3d28; no voters:
I20260619 06:55:20.207499 3341 leader_election.cc:288] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:20.207641 3346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:20.208317 3346 raft_consensus.cc:686] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [term 1 LEADER]: Becoming Leader. State: Replica: f441d84acc464817915584640d2e3d28, State: Running, Role: LEADER
I20260619 06:55:20.208900 3346 consensus_queue.cc:237] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } }
I20260619 06:55:20.209666 3341 sys_catalog.cc:564] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:20.211926 3348 sys_catalog.cc:455] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f441d84acc464817915584640d2e3d28. Latest consensus state: current_term: 1 leader_uuid: "f441d84acc464817915584640d2e3d28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } } }
I20260619 06:55:20.212584 3348 sys_catalog.cc:458] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:20.212953 3347 sys_catalog.cc:455] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f441d84acc464817915584640d2e3d28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f441d84acc464817915584640d2e3d28" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35377 } } }
I20260619 06:55:20.213460 3347 sys_catalog.cc:458] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:20.215658 3353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:20.220907 3353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:20.230814 3353 catalog_manager.cc:1347] Generated new cluster ID: 30cfd38068f649c5b34156e6202199f3
I20260619 06:55:20.230952 3353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:20.244640 3353 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:20.246310 3353 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:20.253283 3353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: Generated new TSK 0
I20260619 06:55:20.254437 3353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:55:20.325150 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:20.671491 3367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:20.671793 3367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:20.671895 3367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:20.680367 3367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:20.680519 3367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:20.680735 3367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:20.691375 3367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:20.692656 3367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:20.694175 3367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:20.707536 3373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:20.707496 3372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:20.707473 3375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:20.708315 3367 server_base.cc:1034] running on GCE node
I20260619 06:55:20.708940 3367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:20.710032 3367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:20.711264 3367 hybrid_clock.cc:648] HybridClock initialized: now 1781852120711224 us; error 43 us; skew 500 ppm
I20260619 06:55:20.711614 3367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:20 dist-test-slave-1h5s krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852120, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:20.715034 3367 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:55:20.716677 3367 webserver.cc:460] Webserver started at http://127.18.39.65:34305/ using document root <none> and password file <none>
I20260619 06:55:20.717465 3367 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:20.717576 3367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:20.717844 3367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:20.720746 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "c75ef887ec28edbb4c81fe68804a1b88"
server_key_iv: "2f2e5a2c072fb3191edef2fbb3c06249"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.721550 3367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "c75ef887ec28edbb4c81fe68804a1b88"
server_key_iv: "2f2e5a2c072fb3191edef2fbb3c06249"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.727149 3367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:20.730804 3382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:20.732214 3367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:20.732383 3367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8"
format_stamp: "Formatted at 2026-06-19 06:55:20 on dist-test-slave-1h5s"
server_key: "c75ef887ec28edbb4c81fe68804a1b88"
server_key_iv: "2f2e5a2c072fb3191edef2fbb3c06249"
server_key_version: "encryptionkey@0"
I20260619 06:55:20.732648 3367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:20.749752 3367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:20.754438 3367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:20.754747 3367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:20.755738 3367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:20.757141 3367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:20.757213 3367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:20.757304 3367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:20.757397 3367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 06:55:20.791415 3367 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:40291
I20260619 06:55:20.791461 3494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:40291 every 8 connection(s)
I20260619 06:55:20.792847 3367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:20.800482 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3367
I20260619 06:55:20.800719 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:20.801419 18589 external_mini_cluster.cc:1428] Setting key ed74d2adc602c79166abd442aa6031a2
Jun 19 06:55:20 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852120, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:20.815846 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:20.795502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:35545 (local address 127.18.39.126:35377)
0619 06:55:20.795955 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:20.795960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:20.796914 (+ 954us) server_negotiation.cc:407] Connection header received
0619 06:55:20.798789 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:20.798793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:20.798876 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:20.798976 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:20.800815 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.801677 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.803697 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.804110 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.804211 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:20.809014 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:20.809025 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:20.809032 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:20.809084 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:20.812549 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:20.813385 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:20.813392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:20.813400 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:20.813519 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:20.813968 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:20.813974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:20.813977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:20.814263 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:20.814416 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:20.815211 (+ 795us) server_negotiation.cc:299] Negotiation successful
0619 06:55:20.815454 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":222,"threads_started":1}
I20260619 06:55:20.816946 3498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:20.796051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35377 (local address 127.18.39.65:35545)
0619 06:55:20.796729 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:20.796775 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:20.798518 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:20.799191 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:20.799199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:20.799761 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:20.800593 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:20.800624 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.801836 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.801842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:20.803544 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:20.803560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:20.804365 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:20.804374 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:20.804721 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:20.805566 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:20.805608 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:20.808721 (+ 3113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:20.812716 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:20.812734 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:20.812754 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:20.813215 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:20.813637 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:20.813643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:20.813646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:20.813850 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:20.814400 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:20.814409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:20.814792 (+ 383us) client_negotiation.cc:769] Sending connection context
0619 06:55:20.815110 (+ 318us) client_negotiation.cc:240] Negotiation successful
0619 06:55:20.815691 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":262,"threads_started":1}
I20260619 06:55:20.818709 3495 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35377
I20260619 06:55:20.819078 3495 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:20.820099 3495 heartbeater.cc:511] Master 127.18.39.126:35377 requested a full tablet report, sending...
I20260619 06:55:20.822741 3306 ts_manager.cc:194] Registered new tserver with Master: 8ed512e2ab12441ab74e4fcabcaf6ea8 (127.18.39.65:40291)
I20260619 06:55:20.824506 3306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:35545
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:55:20.860188 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:21.197257 3501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:21.197650 3501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:21.197856 3501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:21.205209 3501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:21.205394 3501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:21.205638 3501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:21.216012 3501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:21.217401 3501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:21.218781 3501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:21.232128 3507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:21.232066 3509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:21.232632 3506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:21.232908 3501 server_base.cc:1034] running on GCE node
I20260619 06:55:21.233714 3501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:21.234779 3501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:21.236011 3501 hybrid_clock.cc:648] HybridClock initialized: now 1781852121235938 us; error 72 us; skew 500 ppm
I20260619 06:55:21.236378 3501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:21 dist-test-slave-1h5s krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852121, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:21.246058 3501 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:55:21.247617 3501 webserver.cc:460] Webserver started at http://127.18.39.66:34057/ using document root <none> and password file <none>
I20260619 06:55:21.248354 3501 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:21.248497 3501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:21.248816 3501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:21.251819 3501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "5e5923b0f559436badaa701ab6a6a51c"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "6fbce0f854fe8283085785760ebd341c"
server_key_iv: "5b58e4df0d8763304935930025b6850b"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.252611 3501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5923b0f559436badaa701ab6a6a51c"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "6fbce0f854fe8283085785760ebd341c"
server_key_iv: "5b58e4df0d8763304935930025b6850b"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.257879 3501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:55:21.261292 3516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.262703 3501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:21.262876 3501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "5e5923b0f559436badaa701ab6a6a51c"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "6fbce0f854fe8283085785760ebd341c"
server_key_iv: "5b58e4df0d8763304935930025b6850b"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.263083 3501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:21.279969 3501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:21.284473 3501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:21.284785 3501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:21.285817 3501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:21.287151 3501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:21.287247 3501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.287393 3501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:21.287462 3501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.321147 3501 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:42219
I20260619 06:55:21.321190 3628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:42219 every 8 connection(s)
I20260619 06:55:21.322487 3501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:21.325074 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3501
I20260619 06:55:21.325295 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:21.325758 18589 external_mini_cluster.cc:1428] Setting key 4596cad27ed4a8a9227daf5c24971e36
Jun 19 06:55:21 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852121, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:21.345580 3631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.325022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:47275 (local address 127.18.39.126:35377)
0619 06:55:21.327691 (+ 2669us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.327699 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.327720 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:55:21.329041 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.329050 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.329141 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.329263 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:21.331068 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.332218 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.334168 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.334955 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.335097 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.339535 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:21.339540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:21.339544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:21.339582 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:21.342318 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.343134 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.343138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.343140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.343275 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.343742 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.343746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.343748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.344030 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:21.344194 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.344913 (+ 719us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.345157 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2528,"thread_start_us":2398,"threads_started":1}
I20260619 06:55:21.346568 3632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.325821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35377 (local address 127.18.39.66:47275)
0619 06:55:21.326785 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:21.326846 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:21.328778 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:21.329619 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:21.329628 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:21.330224 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:21.330871 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.330892 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.332391 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.332395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.333985 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.334000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.335202 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.335208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.335432 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.336247 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:21.336287 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:21.339251 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:21.342487 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:21.342498 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:21.342517 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:21.342981 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:21.343390 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:21.343395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:21.343399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:21.343614 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:21.344173 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:21.344182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:21.344549 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 06:55:21.344823 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 06:55:21.345393 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":369,"threads_started":1}
I20260619 06:55:21.348222 3629 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35377
I20260619 06:55:21.348551 3629 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:21.349326 3629 heartbeater.cc:511] Master 127.18.39.126:35377 requested a full tablet report, sending...
I20260619 06:55:21.351106 3306 ts_manager.cc:194] Registered new tserver with Master: 5e5923b0f559436badaa701ab6a6a51c (127.18.39.66:42219)
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
I20260619 06:55:21.352020 3306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:47275
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:55:21.384786 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:21.719233 3635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:21.719558 3635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:21.719723 3635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:21.727260 3635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:21.727421 3635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:21.727602 3635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:21.738034 3635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:21.739430 3635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:21.740712 3635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:21.754498 3640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:21.754426 3643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:21.754472 3641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:21.755518 3635 server_base.cc:1034] running on GCE node
I20260619 06:55:21.756132 3635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:21.757050 3635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:21.758273 3635 hybrid_clock.cc:648] HybridClock initialized: now 1781852121758206 us; error 55 us; skew 500 ppm
I20260619 06:55:21.758631 3635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:21 dist-test-slave-1h5s krb5kdc[3261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852121, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:21.762161 3635 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:55:21.763818 3635 webserver.cc:460] Webserver started at http://127.18.39.67:44261/ using document root <none> and password file <none>
I20260619 06:55:21.764490 3635 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:21.764600 3635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:21.764869 3635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:21.768040 3635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "4a350b1fa53851d5a380290b7a5a0620"
server_key_iv: "4327754e5d5c8235db0487c4fcc1ddb6"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.768812 3635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "4a350b1fa53851d5a380290b7a5a0620"
server_key_iv: "4327754e5d5c8235db0487c4fcc1ddb6"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.774415 3635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260619 06:55:21.777985 3650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.779560 3635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:21.779733 3635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425"
format_stamp: "Formatted at 2026-06-19 06:55:21 on dist-test-slave-1h5s"
server_key: "4a350b1fa53851d5a380290b7a5a0620"
server_key_iv: "4327754e5d5c8235db0487c4fcc1ddb6"
server_key_version: "encryptionkey@0"
I20260619 06:55:21.779913 3635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:21.804862 3635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:21.808979 3635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:21.809233 3635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:21.810328 3635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:21.811666 3635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:21.811739 3635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.811823 3635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:21.811867 3635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:21.827716 3495 heartbeater.cc:503] Master 127.18.39.126:35377 was elected leader, sending a full tablet report...
I20260619 06:55:21.850039 3635 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:34101
I20260619 06:55:21.850080 3762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:34101 every 8 connection(s)
I20260619 06:55:21.851325 3635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:21.860160 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3635
I20260619 06:55:21.860358 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:21.860844 18589 external_mini_cluster.cc:1428] Setting key 601f21358f127bff89aa032150702c0a
Jun 19 06:55:21 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852121, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:21.871932 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.853891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:35231 (local address 127.18.39.126:35377)
0619 06:55:21.854369 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.854376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.855416 (+ 1040us) server_negotiation.cc:407] Connection header received
0619 06:55:21.857226 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.857231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.857304 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.857467 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:21.858961 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.859808 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.861602 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.862206 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.862324 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.866218 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:21.866223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:21.866226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:21.866259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:21.868793 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.869596 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.869604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.869608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.869734 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.870169 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.870174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.870178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.870459 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:21.870623 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.871327 (+ 704us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.871551 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":211,"threads_started":1}
I20260619 06:55:21.872912 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.854524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35377 (local address 127.18.39.67:35231)
0619 06:55:21.855244 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:21.855282 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:21.856945 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:21.857672 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:21.857681 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:21.858215 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:21.858793 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.858810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.859955 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.859959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.861426 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.861450 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.862428 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.862434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.862657 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.863438 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:21.863470 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:21.866000 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:21.868942 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:21.868952 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:21.868970 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:21.869480 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:21.869845 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:21.869850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:21.869854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:21.870049 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:21.870595 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:21.870603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:21.870955 (+ 352us) client_negotiation.cc:769] Sending connection context
0619 06:55:21.871228 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 06:55:21.871774 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260619 06:55:21.874318 3763 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35377
I20260619 06:55:21.874660 3763 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:21.875486 3763 heartbeater.cc:511] Master 127.18.39.126:35377 requested a full tablet report, sending...
I20260619 06:55:21.876953 3306 ts_manager.cc:194] Registered new tserver with Master: d06a0a1c7fde41c1ba00c8c4c03fc425 (127.18.39.67:34101)
I20260619 06:55:21.877836 3306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:35231
I20260619 06:55:21.889093 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:21.906559 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.893865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.18.39.126:35377)
0619 06:55:21.894105 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.894115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.894446 (+ 331us) server_negotiation.cc:407] Connection header received
0619 06:55:21.894573 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.894577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.894647 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.894729 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:21.896188 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.896904 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.898304 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.898709 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.898808 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.901217 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:21.901222 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:21.901225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:21.901259 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:21.903986 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.904602 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.904605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.904608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.904718 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:21.905030 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:21.905032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:21.905034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:21.905407 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:21.905589 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.905948 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.906205 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:55:21.910647 3306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:21.917968 3306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:21.941643 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.930927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.18.39.67:34101)
0619 06:55:21.933877 (+ 2950us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.933885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.933915 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:55:21.934010 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.934018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.934249 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.934771 (+ 522us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:21.935331 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.936908 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.939079 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.940296 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.940450 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.940703 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.941064 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.941237 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2775,"thread_start_us":2610,"threads_started":1}
I20260619 06:55:21.941743 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.930638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:34101 (local address 127.0.0.1:57232)
0619 06:55:21.931400 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:21.931451 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:21.931650 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:21.934457 (+ 2807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:21.934462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:21.934484 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:21.935096 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.935117 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.937053 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.937058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.938880 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.938894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.940533 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.940547 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.940821 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.940861 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 06:55:21.941012 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 06:55:21.941228 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":32,"thread_start_us":330,"threads_started":1}
I20260619 06:55:21.941826 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.931207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:42219 (local address 127.0.0.1:60068)
0619 06:55:21.932310 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:21.932334 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:21.932467 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:21.933641 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:21.933651 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:21.933684 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:21.934052 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.934064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.936201 (+ 2137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.936206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.938852 (+ 2646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.938870 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.940843 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.940850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.941023 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.941066 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 06:55:21.941137 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 06:55:21.941306 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":4,"thread_start_us":823,"threads_started":1}
I20260619 06:55:21.942881 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.931714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.18.39.66:42219)
0619 06:55:21.933011 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.933022 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.933052 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:55:21.933136 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.933141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.933326 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.934248 (+ 922us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:21.934482 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.936051 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.939047 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.940594 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.942091 (+ 1497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.942294 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.942369 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.942538 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1154,"thread_start_us":1041,"threads_started":1}
I20260619 06:55:21.945294 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.932744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60422 (local address 127.18.39.65:40291)
0619 06:55:21.934277 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:21.934288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:21.935752 (+ 1464us) server_negotiation.cc:407] Connection header received
0619 06:55:21.935986 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:21.935994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:21.936226 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:21.936398 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:21.937236 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.938841 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.941422 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.944105 (+ 2683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.944262 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.944486 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:21.944781 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 06:55:21.944952 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1333,"thread_start_us":1162,"threads_started":1}
I20260619 06:55:21.945392 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:21.932015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60422)
0619 06:55:21.935566 (+ 3551us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:21.935583 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:21.935807 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:21.936522 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:21.936529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:21.936567 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:21.937047 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.937058 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.939020 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.939025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.941205 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:21.941217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:21.944363 (+ 3146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:21.944371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:21.944566 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:21.944604 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:55:21.944728 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 06:55:21.944896 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3427,"mutex_wait_us":1,"thread_start_us":3278,"threads_started":1}
I20260619 06:55:21.946365 3698 tablet_service.cc:1467] Processing CreateTablet for tablet 9eca1ca81f89408880e2c075c2b63047 (DEFAULT_TABLE table=test-table [id=f4467320a17940ba93a5a1cb9e10c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:21.946883 3564 tablet_service.cc:1467] Processing CreateTablet for tablet 9eca1ca81f89408880e2c075c2b63047 (DEFAULT_TABLE table=test-table [id=f4467320a17940ba93a5a1cb9e10c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:21.948053 3698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eca1ca81f89408880e2c075c2b63047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:21.948496 3564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eca1ca81f89408880e2c075c2b63047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:21.949540 3430 tablet_service.cc:1467] Processing CreateTablet for tablet 9eca1ca81f89408880e2c075c2b63047 (DEFAULT_TABLE table=test-table [id=f4467320a17940ba93a5a1cb9e10c1d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:21.951077 3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eca1ca81f89408880e2c075c2b63047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:21.960928 3782 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Bootstrap starting.
I20260619 06:55:21.964138 3782 tablet_bootstrap.cc:654] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:21.965255 3783 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Bootstrap starting.
I20260619 06:55:21.965523 3782 log.cc:824] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:21.967873 3785 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Bootstrap starting.
I20260619 06:55:21.969715 3783 tablet_bootstrap.cc:654] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:21.971159 3785 tablet_bootstrap.cc:654] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:21.971350 3783 log.cc:824] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:21.972647 3785 log.cc:824] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:21.974489 3783 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: No bootstrap required, opened a new log
I20260619 06:55:21.974493 3782 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: No bootstrap required, opened a new log
I20260619 06:55:21.974802 3783 ts_tablet_manager.cc:1397] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 06:55:21.974802 3782 ts_tablet_manager.cc:1397] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260619 06:55:21.975627 3785 tablet_bootstrap.cc:492] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: No bootstrap required, opened a new log
I20260619 06:55:21.976014 3785 ts_tablet_manager.cc:1397] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260619 06:55:21.984611 3785 raft_consensus.cc:348] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.985074 3785 raft_consensus.cc:374] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:21.985200 3785 raft_consensus.cc:729] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed512e2ab12441ab74e4fcabcaf6ea8, State: Initialized, Role: FOLLOWER
I20260619 06:55:21.985955 3785 consensus_queue.cc:260] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.986053 3782 raft_consensus.cc:348] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.986656 3782 raft_consensus.cc:374] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:21.986802 3782 raft_consensus.cc:729] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06a0a1c7fde41c1ba00c8c4c03fc425, State: Initialized, Role: FOLLOWER
I20260619 06:55:21.986608 3783 raft_consensus.cc:348] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.987118 3783 raft_consensus.cc:374] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:21.987274 3783 raft_consensus.cc:729] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5923b0f559436badaa701ab6a6a51c, State: Initialized, Role: FOLLOWER
I20260619 06:55:21.987636 3782 consensus_queue.cc:260] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.989117 3785 ts_tablet_manager.cc:1428] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260619 06:55:21.989248 3783 consensus_queue.cc:260] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:21.990460 3763 heartbeater.cc:503] Master 127.18.39.126:35377 was elected leader, sending a full tablet report...
I20260619 06:55:21.991425 3782 ts_tablet_manager.cc:1428] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260619 06:55:21.992434 3629 heartbeater.cc:503] Master 127.18.39.126:35377 was elected leader, sending a full tablet report...
I20260619 06:55:21.994310 3783 ts_tablet_manager.cc:1428] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
W20260619 06:55:22.045269 3496 tablet.cc:2369] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:55:22.074571 3630 tablet.cc:2369] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:55:22.102952 3764 tablet.cc:2369] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:22.255503 3789 raft_consensus.cc:482] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:22.255805 3789 raft_consensus.cc:504] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:22.257601 3789 leader_election.cc:288] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ed512e2ab12441ab74e4fcabcaf6ea8 (127.18.39.65:40291), 5e5923b0f559436badaa701ab6a6a51c (127.18.39.66:42219)
I20260619 06:55:22.263142 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.257981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:41681 (local address 127.18.39.65:40291)
0619 06:55:22.258209 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.258218 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.258244 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:55:22.258496 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.258505 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.258591 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.258687 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:22.259100 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.259833 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.261491 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.262271 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.262444 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.262529 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.262772 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.262898 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:55:22.263160 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.257739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.18.39.67:41681)
0619 06:55:22.258099 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.258121 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:22.258254 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:22.258737 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:22.258743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:22.258780 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:22.258943 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:22.258951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.259964 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.259972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:22.261284 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:22.261299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.262430 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.262436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:22.262615 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.262640 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:55:22.262746 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 06:55:22.262875 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260619 06:55:22.263944 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.258207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:54319 (local address 127.18.39.66:42219)
0619 06:55:22.258460 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.258469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.258993 (+ 524us) server_negotiation.cc:407] Connection header received
0619 06:55:22.259183 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.259192 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.259248 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.259345 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:22.259900 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.260600 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.262254 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.262995 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.263193 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.263264 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.263577 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.263734 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:55:22.264463 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.258021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:42219 (local address 127.18.39.67:54319)
0619 06:55:22.258890 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.258906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:22.259040 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:22.259361 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:22.259365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:22.259387 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:22.259736 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:22.259745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.260771 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.260775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:22.262104 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:22.262115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.263199 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.263205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:22.263387 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.263425 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:55:22.264125 (+ 700us) client_negotiation.cc:240] Negotiation successful
0619 06:55:22.264230 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":444,"threads_started":1}
I20260619 06:55:22.264309 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eca1ca81f89408880e2c075c2b63047" candidate_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" is_pre_election: true
I20260619 06:55:22.264905 3450 raft_consensus.cc:2454] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d06a0a1c7fde41c1ba00c8c4c03fc425 in term 0.
I20260619 06:55:22.265467 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eca1ca81f89408880e2c075c2b63047" candidate_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5923b0f559436badaa701ab6a6a51c" is_pre_election: true
I20260619 06:55:22.265729 3653 leader_election.cc:302] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ed512e2ab12441ab74e4fcabcaf6ea8, d06a0a1c7fde41c1ba00c8c4c03fc425; no voters:
I20260619 06:55:22.266048 3584 raft_consensus.cc:2454] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d06a0a1c7fde41c1ba00c8c4c03fc425 in term 0.
I20260619 06:55:22.266122 3789 raft_consensus.cc:2781] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:22.266268 3789 raft_consensus.cc:482] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:22.266381 3789 raft_consensus.cc:3037] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:22.268846 3789 raft_consensus.cc:504] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:22.269527 3789 leader_election.cc:288] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [CANDIDATE]: Term 1 election: Requested vote from peers 8ed512e2ab12441ab74e4fcabcaf6ea8 (127.18.39.65:40291), 5e5923b0f559436badaa701ab6a6a51c (127.18.39.66:42219)
I20260619 06:55:22.269973 3450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eca1ca81f89408880e2c075c2b63047" candidate_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8"
I20260619 06:55:22.269973 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eca1ca81f89408880e2c075c2b63047" candidate_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5923b0f559436badaa701ab6a6a51c"
I20260619 06:55:22.270191 3450 raft_consensus.cc:3037] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:22.270191 3584 raft_consensus.cc:3037] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:22.273473 3584 raft_consensus.cc:2454] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d06a0a1c7fde41c1ba00c8c4c03fc425 in term 1.
I20260619 06:55:22.273464 3450 raft_consensus.cc:2454] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d06a0a1c7fde41c1ba00c8c4c03fc425 in term 1.
I20260619 06:55:22.273923 3652 leader_election.cc:302] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e5923b0f559436badaa701ab6a6a51c, d06a0a1c7fde41c1ba00c8c4c03fc425; no voters:
I20260619 06:55:22.274262 3789 raft_consensus.cc:2781] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:22.274725 3789 raft_consensus.cc:686] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 1 LEADER]: Becoming Leader. State: Replica: d06a0a1c7fde41c1ba00c8c4c03fc425, State: Running, Role: LEADER
I20260619 06:55:22.275194 3789 consensus_queue.cc:237] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } }
I20260619 06:55:22.279723 3304 catalog_manager.cc:5495] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 reported cstate change: term changed from 0 to 1, leader changed from <none> to d06a0a1c7fde41c1ba00c8c4c03fc425 (127.18.39.67). New cstate: current_term: 1 leader_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a0a1c7fde41c1ba00c8c4c03fc425" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260619 06:55:22.316090 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.310601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.18.39.67:34101)
0619 06:55:22.310816 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.310823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.310843 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:55:22.310983 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.310987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.311064 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.311224 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:22.311623 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.312323 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.313817 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.314217 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.314343 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.314857 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:22.315455 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:22.315610 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.315709 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.315794 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:22.325011 3584 raft_consensus.cc:1261] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 1 FOLLOWER]: Refusing update from remote peer d06a0a1c7fde41c1ba00c8c4c03fc425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:22.325047 3450 raft_consensus.cc:1261] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 1 FOLLOWER]: Refusing update from remote peer d06a0a1c7fde41c1ba00c8c4c03fc425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:22.326193 3792 consensus_queue.cc:1035] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed512e2ab12441ab74e4fcabcaf6ea8" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:22.326637 3789 consensus_queue.cc:1035] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5923b0f559436badaa701ab6a6a51c" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:22.339704 3795 mvcc.cc:205] Tried to move back new op lower bound from 7298466293031395328 to 7298466292842598400. Current Snapshot: MvccSnapshot[applied={T|T < 7298466293031395328}]
I20260619 06:55:22.354966 3306 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:22.355289 3306 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:22.360496 3306 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f441d84acc464817915584640d2e3d28: Sending DeleteTablet for 3 replicas of tablet 9eca1ca81f89408880e2c075c2b63047
I20260619 06:55:22.362742 3698 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eca1ca81f89408880e2c075c2b63047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:22 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:57232
I20260619 06:55:22.363078 3430 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eca1ca81f89408880e2c075c2b63047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:22 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60422
I20260619 06:55:22.363802 3564 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eca1ca81f89408880e2c075c2b63047 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:22 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60068
I20260619 06:55:22.365805 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3367
I20260619 06:55:22.376649 3804 tablet_replica.cc:330] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: stopping tablet replica
I20260619 06:55:22.377315 3805 tablet_replica.cc:330] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: stopping tablet replica
I20260619 06:55:22.377997 3805 raft_consensus.cc:2229] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:22.378528 3804 raft_consensus.cc:2229] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:22.379313 3804 raft_consensus.cc:2258] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:22.379907 3805 raft_consensus.cc:2258] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:22.380153 3806 tablet_replica.cc:330] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: stopping tablet replica
I20260619 06:55:22.380592 3806 raft_consensus.cc:2229] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:22.381119 3806 raft_consensus.cc:2258] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:22.382927 3805 ts_tablet_manager.cc:1905] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:22.383131 3806 ts_tablet_manager.cc:1905] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:22.384754 3804 ts_tablet_manager.cc:1905] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 06:55:22 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:55:22.390872 3806 ts_tablet_manager.cc:1918] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:22.391079 3806 log.cc:1196] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/9eca1ca81f89408880e2c075c2b63047
I20260619 06:55:22.391661 3806 ts_tablet_manager.cc:1939] T 9eca1ca81f89408880e2c075c2b63047 P 5e5923b0f559436badaa701ab6a6a51c: Deleting consensus metadata
I20260619 06:55:22.392267 3804 ts_tablet_manager.cc:1918] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:22.393795 3291 catalog_manager.cc:4841] TS 5e5923b0f559436badaa701ab6a6a51c (127.18.39.66:42219): tablet 9eca1ca81f89408880e2c075c2b63047 (table test-table [id=f4467320a17940ba93a5a1cb9e10c1d2]) successfully deleted
I20260619 06:55:22.397511 3804 log.cc:1196] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/9eca1ca81f89408880e2c075c2b63047
I20260619 06:55:22.397714 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.367726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.18.39.65:40291)
0619 06:55:22.367969 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.367978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.372211 (+ 4233us) server_negotiation.cc:407] Connection header received
0619 06:55:22.372398 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.372408 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.372573 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.372717 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:22.379686 (+ 6969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.380948 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.382991 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.383596 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.383759 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.389737 (+ 5978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:22.389756 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:22.389777 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:22.389827 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:22.394375 (+ 4548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.395157 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.395166 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.395168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.395310 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.395726 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.395730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.395732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.396109 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:22.396436 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.396778 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.397028 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:55:22.398316 3804 ts_tablet_manager.cc:1939] T 9eca1ca81f89408880e2c075c2b63047 P d06a0a1c7fde41c1ba00c8c4c03fc425: Deleting consensus metadata
I20260619 06:55:22.398726 3490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:22.400662 3292 catalog_manager.cc:4841] TS d06a0a1c7fde41c1ba00c8c4c03fc425 (127.18.39.67:34101): tablet 9eca1ca81f89408880e2c075c2b63047 (table test-table [id=f4467320a17940ba93a5a1cb9e10c1d2]) successfully deleted
I20260619 06:55:22.492246 3805 ts_tablet_manager.cc:1918] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:22.492552 3805 log.cc:1196] T 9eca1ca81f89408880e2c075c2b63047 P 8ed512e2ab12441ab74e4fcabcaf6ea8: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/9eca1ca81f89408880e2c075c2b63047
I20260619 06:55:22.492985 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3367
W20260619 06:55:22.509763 3292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.39.65:40291 (error 108)
I20260619 06:55:22.510550 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3501
I20260619 06:55:22.512287 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.511446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60434)
0619 06:55:22.511941 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.512070 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":260,"threads_started":1}
W20260619 06:55:22.512575 3292 catalog_manager.cc:4584] TS 8ed512e2ab12441ab74e4fcabcaf6ea8 (127.18.39.65:40291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9eca1ca81f89408880e2c075c2b63047: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Jun 19 06:55:22 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:55:22.526587 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.511381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.18.39.66:42219)
0619 06:55:22.511579 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.511585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.511881 (+ 296us) server_negotiation.cc:407] Connection header received
0619 06:55:22.511956 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.511960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.512033 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.512153 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:22.513606 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.514324 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.515623 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.516071 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.516207 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.520551 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:22.520565 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:22.520581 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:22.520622 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:22.523452 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.524241 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.524248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.524250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.524434 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.524813 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.524820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.524823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.525294 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:22.525625 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.525975 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.526223 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:55:22.527235 3624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:22.566406 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.565878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60440)
0619 06:55:22.566117 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.566231 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 06:55:22.567603 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.567156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60452)
0619 06:55:22.567334 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.567435 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:55:22.619124 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3501
I20260619 06:55:22.628396 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.627820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60464)
0619 06:55:22.628067 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.628201 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 06:55:22.629761 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.629249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60474)
0619 06:55:22.629459 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.629558 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 06:55:22.636700 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3635
Jun 19 06:55:22 dist-test-slave-1h5s krb5kdc[3261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:55:22.654968 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.637580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.18.39.67:34101)
0619 06:55:22.637809 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:22.637816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:22.637838 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:55:22.637910 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:22.637915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:22.638004 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:22.638135 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:22.639353 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.640667 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.642028 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:22.642730 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:22.642863 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:22.647401 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:22.647419 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:22.647439 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:22.647492 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:22.651662 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.652470 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.652480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.652484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.652701 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:22.653109 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:22.653114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:22.653117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:22.653615 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:22.653992 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:22.654303 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 06:55:22.654535 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:55:22.655841 3758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:22.710968 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.710469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60490)
0619 06:55:22.710667 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.710809 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 06:55:22.712110 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.711678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60496)
0619 06:55:22.711880 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.711952 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 06:55:22.782137 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3635
I20260619 06:55:22.801205 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3274
I20260619 06:55:22.802440 3336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:22.915495 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:22.914874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40291 (local address 127.0.0.1:60512)
0619 06:55:22.915108 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:22.915258 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:40291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:55:22.915827 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3274
2026-06-19T06:55:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4591 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T06:55:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:22Z Disabled control of system clock
I20260619 06:55:22.993294 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42347
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:42347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:23.333931 3817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:23.334223 3817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:23.334355 3817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:23.341818 3817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:23.341933 3817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:23.342005 3817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:23.342097 3817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:23.342190 3817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:23.351810 3817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:42347
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:23.353281 3817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:23.355149 3817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:23.367269 3822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:23.367235 3825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:23.367194 3823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:23.368050 3817 server_base.cc:1034] running on GCE node
I20260619 06:55:23.368976 3817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:23.370581 3817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:23.371788 3817 hybrid_clock.cc:648] HybridClock initialized: now 1781852123371749 us; error 39 us; skew 500 ppm
I20260619 06:55:23.372171 3817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:23.374742 3817 webserver.cc:460] Webserver started at http://127.18.39.126:42273/ using document root <none> and password file <none>
I20260619 06:55:23.375496 3817 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:23.375597 3817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:23.375924 3817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:23.379107 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "739842fadebe47b590a8f3c213f0b1fb"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "73cf78d5fec961d1b2f9955c21d2226b"
server_key_iv: "d24e925f6c44a4405f1da20ba98d6daf"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.379984 3817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "739842fadebe47b590a8f3c213f0b1fb"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "73cf78d5fec961d1b2f9955c21d2226b"
server_key_iv: "d24e925f6c44a4405f1da20ba98d6daf"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.385553 3817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:55:23.389184 3831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:23.390679 3817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:55:23.390882 3817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "739842fadebe47b590a8f3c213f0b1fb"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "73cf78d5fec961d1b2f9955c21d2226b"
server_key_iv: "d24e925f6c44a4405f1da20ba98d6daf"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.391111 3817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:23.423480 3817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:23.424398 3817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:23.424713 3817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:23.444015 3817 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:42347
I20260619 06:55:23.444051 3882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:42347 every 8 connection(s)
I20260619 06:55:23.445575 3817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:23.449558 3883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:23.450316 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3817
I20260619 06:55:23.450500 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:23.451123 18589 external_mini_cluster.cc:1428] Setting key 59e552ffd4e34bfb98d3bf760bf80841
I20260619 06:55:23.463100 3883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: Bootstrap starting.
I20260619 06:55:23.465741 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:23.453275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.18.39.126:42347)
0619 06:55:23.454527 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:23.454538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:23.454578 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:55:23.456424 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:23.456448 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:23.457048 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:23.457505 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:23.458092 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:23.459417 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:23.460947 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:23.461756 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:23.461982 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:23.463249 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:23.463262 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:23.463281 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:23.463886 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:23.463970 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:23.463993 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:23.464140 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:23.464366 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 06:55:23.464705 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":509,"threads_started":1}
I20260619 06:55:23.466660 3883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:23.467762 3883 log.cc:824] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:23.470762 3883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: No bootstrap required, opened a new log
I20260619 06:55:23.478552 3883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } }
I20260619 06:55:23.478981 3883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:23.479089 3883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 739842fadebe47b590a8f3c213f0b1fb, State: Initialized, Role: FOLLOWER
I20260619 06:55:23.479645 3883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } }
I20260619 06:55:23.479907 3883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:23.480049 3883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:23.480221 3883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:23.482543 3883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } }
I20260619 06:55:23.483137 3883 leader_election.cc:302] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 739842fadebe47b590a8f3c213f0b1fb; no voters:
I20260619 06:55:23.484073 3883 leader_election.cc:288] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:23.484231 3888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:23.485095 3888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [term 1 LEADER]: Becoming Leader. State: Replica: 739842fadebe47b590a8f3c213f0b1fb, State: Running, Role: LEADER
I20260619 06:55:23.485688 3888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } }
I20260619 06:55:23.486462 3883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:23.488878 3889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "739842fadebe47b590a8f3c213f0b1fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } } }
I20260619 06:55:23.488909 3890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 739842fadebe47b590a8f3c213f0b1fb. Latest consensus state: current_term: 1 leader_uuid: "739842fadebe47b590a8f3c213f0b1fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "739842fadebe47b590a8f3c213f0b1fb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42347 } } }
I20260619 06:55:23.489400 3889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:23.489398 3890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:23.492391 3894 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:23.497967 3894 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:23.507282 3894 catalog_manager.cc:1347] Generated new cluster ID: 5ce4a70718b04a6da6394275b11a8da2
I20260619 06:55:23.507422 3894 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:23.519258 3894 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:23.520570 3894 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:23.527457 3894 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: Generated new TSK 0
I20260619 06:55:23.528492 3894 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:23.539873 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--builtin_ntp_servers=127.18.39.84:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:23.887938 3907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:23.888301 3907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:23.888463 3907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:23.896522 3907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:23.896679 3907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:23.896862 3907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:23.906312 3907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:23.907703 3907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:23.909600 3907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:23.923168 3912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:23.923112 3913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:23.923076 3915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:23.924088 3907 server_base.cc:1034] running on GCE node
I20260619 06:55:23.924789 3907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:23.925886 3907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:23.927141 3907 hybrid_clock.cc:648] HybridClock initialized: now 1781852123927076 us; error 65 us; skew 500 ppm
I20260619 06:55:23.927472 3907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:23.930028 3907 webserver.cc:460] Webserver started at http://127.18.39.65:36841/ using document root <none> and password file <none>
I20260619 06:55:23.930755 3907 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:23.930891 3907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:23.931221 3907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:23.934356 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "922dab4ad88a4eb9903cec1ff3d6c983"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "2f2fdd0bfe29c3c7ed5ae21543078ce5"
server_key_iv: "1d93eaf12ca79789924e7fd0b1da96d4"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.935209 3907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "922dab4ad88a4eb9903cec1ff3d6c983"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "2f2fdd0bfe29c3c7ed5ae21543078ce5"
server_key_iv: "1d93eaf12ca79789924e7fd0b1da96d4"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.941073 3907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 06:55:23.944738 3921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:23.946148 3907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:55:23.946341 3907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "922dab4ad88a4eb9903cec1ff3d6c983"
format_stamp: "Formatted at 2026-06-19 06:55:23 on dist-test-slave-1h5s"
server_key: "2f2fdd0bfe29c3c7ed5ae21543078ce5"
server_key_iv: "1d93eaf12ca79789924e7fd0b1da96d4"
server_key_version: "encryptionkey@0"
I20260619 06:55:23.946593 3907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:23.961894 3907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:23.962916 3907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:23.963228 3907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:23.964262 3907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:23.965975 3907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:23.966101 3907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:23.966250 3907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:23.966338 3907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.001102 3907 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:32967
I20260619 06:55:24.001206 4033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:32967 every 8 connection(s)
I20260619 06:55:24.002473 3907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:24.008024 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 3907
I20260619 06:55:24.008226 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:24.008685 18589 external_mini_cluster.cc:1428] Setting key 0505f721d403e9edc770c83f692da6cf
I20260619 06:55:24.011857 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--builtin_ntp_servers=127.18.39.84:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:24.016762 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.005137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:34293 (local address 127.18.39.126:42347)
0619 06:55:24.005630 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:24.005635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:24.006699 (+ 1064us) server_negotiation.cc:407] Connection header received
0619 06:55:24.008621 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:24.008627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:24.008731 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:24.008867 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:24.010259 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.011213 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.013862 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.014255 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.014351 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.015998 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:24.016003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:24.016007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:24.016026 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:24.016065 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:24.016076 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:24.016145 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:24.016419 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 06:55:24.016499 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":277,"threads_started":1}
I20260619 06:55:24.018330 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.005673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42347 (local address 127.18.39.65:34293)
0619 06:55:24.006532 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:24.006566 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:24.008293 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:24.009165 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:24.009178 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:24.009302 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:24.010091 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.010111 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.011378 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.011385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.013710 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.013729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.014483 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.014492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.014829 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.015610 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:24.015616 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:24.015674 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:24.015784 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:24.016184 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:24.016198 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:24.016209 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:55:24.016392 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 06:55:24.016839 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":257,"threads_started":1}
I20260619 06:55:24.019974 4034 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42347
I20260619 06:55:24.020295 4034 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:24.021301 4034 heartbeater.cc:511] Master 127.18.39.126:42347 requested a full tablet report, sending...
I20260619 06:55:24.023520 3848 ts_manager.cc:194] Registered new tserver with Master: 922dab4ad88a4eb9903cec1ff3d6c983 (127.18.39.65:32967)
I20260619 06:55:24.025202 3848 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.65:34293
W20260619 06:55:24.347697 4038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:24.348039 4038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:24.348199 4038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:24.358592 4038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:24.358743 4038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:24.358952 4038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:24.369068 4038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:24.370442 4038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:24.372187 4038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:24.385900 4043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:24.385969 4044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:24.385957 4046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:24.390956 4038 server_base.cc:1034] running on GCE node
I20260619 06:55:24.391676 4038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:24.392796 4038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:24.394109 4038 hybrid_clock.cc:648] HybridClock initialized: now 1781852124394043 us; error 34 us; skew 500 ppm
I20260619 06:55:24.394443 4038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:24.397053 4038 webserver.cc:460] Webserver started at http://127.18.39.66:37093/ using document root <none> and password file <none>
I20260619 06:55:24.397853 4038 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:24.397989 4038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:24.398386 4038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:24.401477 4038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "59aca7b37813485880e105548283c319"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "8c95a44228fe7946d0849182a6fd3312"
server_key_iv: "047f02cc80849c63a8fb4f3bf325f5c9"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.402318 4038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "59aca7b37813485880e105548283c319"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "8c95a44228fe7946d0849182a6fd3312"
server_key_iv: "047f02cc80849c63a8fb4f3bf325f5c9"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.407742 4038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 06:55:24.411433 4052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.412830 4038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:24.413017 4038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "59aca7b37813485880e105548283c319"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "8c95a44228fe7946d0849182a6fd3312"
server_key_iv: "047f02cc80849c63a8fb4f3bf325f5c9"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.413247 4038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:24.454694 4038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:24.455684 4038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:24.455996 4038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:24.457132 4038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:24.458959 4038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:24.459062 4038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.459162 4038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:24.459230 4038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.493700 4038 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:43667
I20260619 06:55:24.493778 4164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:43667 every 8 connection(s)
I20260619 06:55:24.495069 4038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:24.500368 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4038
I20260619 06:55:24.500590 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:24.501046 18589 external_mini_cluster.cc:1428] Setting key a6bf8e6802d4536cfaaebba88cd71938
I20260619 06:55:24.503930 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--builtin_ntp_servers=127.18.39.84:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:24.508167 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.497756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:54683 (local address 127.18.39.126:42347)
0619 06:55:24.497937 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:24.497941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:24.499367 (+ 1426us) server_negotiation.cc:407] Connection header received
0619 06:55:24.501386 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:24.501392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:24.501485 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:24.501608 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:24.502903 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.503600 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.505269 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.505739 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.505881 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.507319 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:24.507324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:24.507327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:24.507345 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:24.507378 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:24.507390 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:24.507489 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:24.507735 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 06:55:24.507841 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:24.509248 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.498288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42347 (local address 127.18.39.66:54683)
0619 06:55:24.499192 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:24.499229 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:24.501061 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:24.501834 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:24.501842 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:24.501936 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:24.502722 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.502748 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.503767 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.503779 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.505124 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.505136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.505968 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.505975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.506221 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.506963 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:24.506966 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:24.507006 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:24.507100 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:24.507488 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:24.507502 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:24.507513 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:55:24.507725 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 06:55:24.508160 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":283,"threads_started":1}
I20260619 06:55:24.510872 4165 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42347
I20260619 06:55:24.511242 4165 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:24.512259 4165 heartbeater.cc:511] Master 127.18.39.126:42347 requested a full tablet report, sending...
I20260619 06:55:24.513787 3848 ts_manager.cc:194] Registered new tserver with Master: 59aca7b37813485880e105548283c319 (127.18.39.66:43667)
I20260619 06:55:24.514617 3848 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.66:54683
W20260619 06:55:24.839073 4168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:24.839426 4168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:24.839596 4168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:24.847111 4168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:24.847267 4168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:24.847452 4168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:24.857446 4168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:42347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:24.858875 4168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:24.860744 4168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:24.874039 4173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:24.874006 4174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:24.873976 4176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:24.875154 4168 server_base.cc:1034] running on GCE node
I20260619 06:55:24.875839 4168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:24.876809 4168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:24.878039 4168 hybrid_clock.cc:648] HybridClock initialized: now 1781852124877993 us; error 40 us; skew 500 ppm
I20260619 06:55:24.878355 4168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:24.880723 4168 webserver.cc:460] Webserver started at http://127.18.39.67:38183/ using document root <none> and password file <none>
I20260619 06:55:24.881498 4168 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:24.881654 4168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:24.881975 4168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:24.884938 4168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "4686d3cdd3a74f9e92769625fcd076f3"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "b0e7762cde5214c61bac99b5b6aa6bd6"
server_key_iv: "904d912a792725d60b9a772e87390d50"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.885768 4168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "4686d3cdd3a74f9e92769625fcd076f3"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "b0e7762cde5214c61bac99b5b6aa6bd6"
server_key_iv: "904d912a792725d60b9a772e87390d50"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.891076 4168 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:55:24.894748 4182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.896219 4168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:24.896356 4168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "4686d3cdd3a74f9e92769625fcd076f3"
format_stamp: "Formatted at 2026-06-19 06:55:24 on dist-test-slave-1h5s"
server_key: "b0e7762cde5214c61bac99b5b6aa6bd6"
server_key_iv: "904d912a792725d60b9a772e87390d50"
server_key_version: "encryptionkey@0"
I20260619 06:55:24.896536 4168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:24.920145 4168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:24.920991 4168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:24.921270 4168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:24.922354 4168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:24.923941 4168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:24.924031 4168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.924121 4168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:24.924177 4168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:24.958079 4168 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:37757
I20260619 06:55:24.958180 4294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:37757 every 8 connection(s)
I20260619 06:55:24.959455 4168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:24.960996 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4168
I20260619 06:55:24.961194 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:24.961856 18589 external_mini_cluster.cc:1428] Setting key 9acd5c06f4783eec3186b39f9c8041fc
I20260619 06:55:24.973315 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.962160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:53211 (local address 127.18.39.126:42347)
0619 06:55:24.962383 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:24.962390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:24.964510 (+ 2120us) server_negotiation.cc:407] Connection header received
0619 06:55:24.966585 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:24.966589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:24.966674 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:24.966767 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:24.968097 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.968896 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.970676 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.971095 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.971201 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.972624 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:24.972629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:24.972632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:24.972652 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:24.972686 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:24.972698 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:24.972770 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:24.972977 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 06:55:24.973063 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:55:24.974462 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.962846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:42347 (local address 127.18.39.67:53211)
0619 06:55:24.964311 (+ 1465us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:24.964372 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:24.966271 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:24.967057 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:24.967067 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:24.967201 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:24.967897 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.967918 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.969056 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.969060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.970493 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:24.970509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.971316 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.971320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:24.971535 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.972268 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:24.972270 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:24.972312 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:24.972402 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:24.972783 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:24.972790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:24.972796 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:55:24.972951 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:55:24.973291 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":587,"threads_started":1}
I20260619 06:55:24.976001 4295 heartbeater.cc:347] Connected to a master server at 127.18.39.126:42347
I20260619 06:55:24.976711 4295 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:24.977633 4295 heartbeater.cc:511] Master 127.18.39.126:42347 requested a full tablet report, sending...
I20260619 06:55:24.979002 3848 ts_manager.cc:194] Registered new tserver with Master: 4686d3cdd3a74f9e92769625fcd076f3 (127.18.39.67:37757)
I20260619 06:55:24.979851 3848 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.67:53211
I20260619 06:55:24.990042 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:25.000768 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:24.994333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.18.39.126:42347)
0619 06:55:24.994565 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:24.994573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:24.994918 (+ 345us) server_negotiation.cc:407] Connection header received
0619 06:55:24.995127 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:24.995132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:24.995220 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:24.995357 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:24.995881 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.996596 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.998107 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:24.998542 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:24.998685 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:24.999926 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:24.999940 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:24.999944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:24.999964 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:25.000000 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:25.000013 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:25.000113 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.000303 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.000407 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:55:25.004866 3848 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:25.012027 3848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:25.028656 4034 heartbeater.cc:503] Master 127.18.39.126:42347 was elected leader, sending a full tablet report...
I20260619 06:55:25.048811 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.025108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43667 (local address 127.0.0.1:33710)
0619 06:55:25.038025 (+ 12917us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.038056 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:25.038270 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:25.042147 (+ 3877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:25.042151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:25.042177 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:25.042538 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.042552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.044173 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.044179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.046536 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.046546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.047834 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.047840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.048019 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.048075 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:55:25.048212 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 06:55:25.048408 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12681,"mutex_wait_us":26,"thread_start_us":13294,"threads_started":1}
I20260619 06:55:25.048933 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.024981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46174)
0619 06:55:25.025831 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.025880 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:25.026080 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:25.040093 (+ 14013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:25.040105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:25.040166 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:25.040793 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.040818 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.043012 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.043020 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.046174 (+ 3154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.046189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.048200 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.048207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.048395 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.048422 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 06:55:25.048503 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 06:55:25.048618 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":55,"thread_start_us":1064,"threads_started":1}
I20260619 06:55:25.048931 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.025724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33710 (local address 127.18.39.66:43667)
0619 06:55:25.041464 (+ 15740us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.041473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.041501 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:55:25.041594 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.041616 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.041868 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.042153 (+ 285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:25.042729 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.044037 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.046715 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.047634 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.047791 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.048052 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.048399 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.048576 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15528,"thread_start_us":15607,"threads_started":1}
I20260619 06:55:25.049620 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.025183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.18.39.65:32967)
0619 06:55:25.039198 (+ 14015us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.039212 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.039255 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:55:25.039385 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.039398 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.039780 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.040053 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:25.041063 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.042875 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.046413 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.047902 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.048033 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.048281 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.049123 (+ 842us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.049264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13768,"thread_start_us":13619,"threads_started":1}
I20260619 06:55:25.051901 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.027929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:37757 (local address 127.0.0.1:45230)
0619 06:55:25.040816 (+ 12887us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.040833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:25.040940 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:25.043400 (+ 2460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:25.043409 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:25.043434 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:25.043846 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.043858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.045610 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.045615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.047684 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.047699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.051017 (+ 3318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.051024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.051208 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.051240 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:55:25.051395 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:55:25.051546 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12734,"mutex_wait_us":50,"thread_start_us":12561,"threads_started":1}
I20260619 06:55:25.051937 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.028254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.18.39.67:37757)
0619 06:55:25.042869 (+ 14615us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.042878 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.042911 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:55:25.042998 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.043005 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.043209 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.043387 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:25.044059 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.045443 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.047916 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.050802 (+ 2886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.050939 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.051209 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.051397 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.051590 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14436,"thread_start_us":14255,"threads_started":1}
I20260619 06:55:25.053478 4100 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed (DEFAULT_TABLE table=test-table [id=604dd88b485b470a8e9491f7cde8cb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:25.053381 3969 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed (DEFAULT_TABLE table=test-table [id=604dd88b485b470a8e9491f7cde8cb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:25.054790 4100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad4361e7e8743549ce6f49f2cbd43ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:25.054798 3969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad4361e7e8743549ce6f49f2cbd43ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:25.055979 4230 tablet_service.cc:1467] Processing CreateTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed (DEFAULT_TABLE table=test-table [id=604dd88b485b470a8e9491f7cde8cb2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:25.057494 4230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ad4361e7e8743549ce6f49f2cbd43ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:25.066299 4313 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Bootstrap starting.
I20260619 06:55:25.069370 4314 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Bootstrap starting.
I20260619 06:55:25.071044 4313 tablet_bootstrap.cc:654] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:25.072507 4313 log.cc:824] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:25.073087 4314 tablet_bootstrap.cc:654] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:25.073163 4315 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Bootstrap starting.
I20260619 06:55:25.074635 4314 log.cc:824] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:25.076733 4315 tablet_bootstrap.cc:654] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:25.077831 4314 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: No bootstrap required, opened a new log
I20260619 06:55:25.078039 4315 log.cc:824] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:25.078122 4314 ts_tablet_manager.cc:1397] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260619 06:55:25.078166 4313 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: No bootstrap required, opened a new log
I20260619 06:55:25.078434 4313 ts_tablet_manager.cc:1397] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260619 06:55:25.081859 4315 tablet_bootstrap.cc:492] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: No bootstrap required, opened a new log
I20260619 06:55:25.082134 4315 ts_tablet_manager.cc:1397] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260619 06:55:25.086072 4314 raft_consensus.cc:348] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.086584 4314 raft_consensus.cc:374] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:25.086679 4314 raft_consensus.cc:729] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922dab4ad88a4eb9903cec1ff3d6c983, State: Initialized, Role: FOLLOWER
I20260619 06:55:25.087361 4314 consensus_queue.cc:260] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.088855 4313 raft_consensus.cc:348] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.089325 4313 raft_consensus.cc:374] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:25.089419 4314 ts_tablet_manager.cc:1428] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260619 06:55:25.089457 4313 raft_consensus.cc:729] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aca7b37813485880e105548283c319, State: Initialized, Role: FOLLOWER
I20260619 06:55:25.090080 4313 consensus_queue.cc:260] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.091866 4313 ts_tablet_manager.cc:1428] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Time spent starting tablet: real 0.013s user 0.007s sys 0.007s
I20260619 06:55:25.092025 4165 heartbeater.cc:503] Master 127.18.39.126:42347 was elected leader, sending a full tablet report...
I20260619 06:55:25.094509 4315 raft_consensus.cc:348] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.095628 4315 raft_consensus.cc:374] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:25.095742 4315 raft_consensus.cc:729] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4686d3cdd3a74f9e92769625fcd076f3, State: Initialized, Role: FOLLOWER
I20260619 06:55:25.096434 4315 consensus_queue.cc:260] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.097494 4295 heartbeater.cc:503] Master 127.18.39.126:42347 was elected leader, sending a full tablet report...
I20260619 06:55:25.098161 4315 ts_tablet_manager.cc:1428] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260619 06:55:25.184985 4319 raft_consensus.cc:482] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:25.185415 4319 raft_consensus.cc:504] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.187243 4319 leader_election.cc:288] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59aca7b37813485880e105548283c319 (127.18.39.66:43667), 4686d3cdd3a74f9e92769625fcd076f3 (127.18.39.67:37757)
I20260619 06:55:25.196947 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.187652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:33625 (local address 127.18.39.66:43667)
0619 06:55:25.187878 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.187884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.188754 (+ 870us) server_negotiation.cc:407] Connection header received
0619 06:55:25.188949 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.188955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.189043 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.189147 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:25.190123 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.191250 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.194585 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.195816 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.196013 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.196091 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.196592 (+ 501us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.196716 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:55:25.196931 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.187723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:52231 (local address 127.18.39.67:37757)
0619 06:55:25.187924 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.187930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.188756 (+ 826us) server_negotiation.cc:407] Connection header received
0619 06:55:25.188952 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.188957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.189043 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.189147 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:25.190124 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.191253 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.194547 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.195910 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.196067 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.196137 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.196538 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.196675 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 06:55:25.197043 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.187647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:37757 (local address 127.18.39.65:52231)
0619 06:55:25.188585 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.188598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:25.188763 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:25.189202 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:25.189208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:25.189258 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:25.189917 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.189937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.191410 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.191416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.194241 (+ 2825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.194258 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.196158 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.196165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.196344 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.196381 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:55:25.196481 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 06:55:25.196628 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":283,"spinlock_wait_cycles":16384,"thread_start_us":447,"threads_started":1}
I20260619 06:55:25.197647 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.187647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:43667 (local address 127.18.39.65:33625)
0619 06:55:25.188561 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.188586 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:25.188802 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:25.189202 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:25.189208 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:25.189259 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:25.189956 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.189973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.191411 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.191416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.194239 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:25.194258 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.196245 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.196250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:25.196375 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.196398 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:55:25.197152 (+ 754us) client_negotiation.cc:240] Negotiation successful
0619 06:55:25.197313 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":176,"spinlock_wait_cycles":49024,"thread_start_us":452,"threads_started":1}
I20260619 06:55:25.198143 4250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad4361e7e8743549ce6f49f2cbd43ed" candidate_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4686d3cdd3a74f9e92769625fcd076f3" is_pre_election: true
I20260619 06:55:25.198673 4250 raft_consensus.cc:2454] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922dab4ad88a4eb9903cec1ff3d6c983 in term 0.
I20260619 06:55:25.198688 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad4361e7e8743549ce6f49f2cbd43ed" candidate_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aca7b37813485880e105548283c319" is_pre_election: true
I20260619 06:55:25.199213 4120 raft_consensus.cc:2454] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 922dab4ad88a4eb9903cec1ff3d6c983 in term 0.
I20260619 06:55:25.199389 3924 leader_election.cc:302] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4686d3cdd3a74f9e92769625fcd076f3, 922dab4ad88a4eb9903cec1ff3d6c983; no voters:
I20260619 06:55:25.199891 4319 raft_consensus.cc:2781] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:25.200055 4319 raft_consensus.cc:482] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:25.200189 4319 raft_consensus.cc:3037] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:25.202447 4319 raft_consensus.cc:504] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
I20260619 06:55:25.203051 4319 leader_election.cc:288] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [CANDIDATE]: Term 1 election: Requested vote from peers 59aca7b37813485880e105548283c319 (127.18.39.66:43667), 4686d3cdd3a74f9e92769625fcd076f3 (127.18.39.67:37757)
I20260619 06:55:25.203557 4250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad4361e7e8743549ce6f49f2cbd43ed" candidate_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4686d3cdd3a74f9e92769625fcd076f3"
I20260619 06:55:25.203559 4120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ad4361e7e8743549ce6f49f2cbd43ed" candidate_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aca7b37813485880e105548283c319"
I20260619 06:55:25.203773 4250 raft_consensus.cc:3037] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:25.203759 4120 raft_consensus.cc:3037] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:25.206108 4250 raft_consensus.cc:2454] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922dab4ad88a4eb9903cec1ff3d6c983 in term 1.
I20260619 06:55:25.206125 4120 raft_consensus.cc:2454] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 922dab4ad88a4eb9903cec1ff3d6c983 in term 1.
I20260619 06:55:25.206609 3924 leader_election.cc:302] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4686d3cdd3a74f9e92769625fcd076f3, 922dab4ad88a4eb9903cec1ff3d6c983; no voters:
I20260619 06:55:25.206939 4319 raft_consensus.cc:2781] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:25.207412 4319 raft_consensus.cc:686] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 1 LEADER]: Becoming Leader. State: Replica: 922dab4ad88a4eb9903cec1ff3d6c983, State: Running, Role: LEADER
I20260619 06:55:25.207901 4319 consensus_queue.cc:237] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } }
W20260619 06:55:25.211109 4296 tablet.cc:2369] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:25.212844 3848 catalog_manager.cc:5495] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 reported cstate change: term changed from 0 to 1, leader changed from <none> to 922dab4ad88a4eb9903cec1ff3d6c983 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "922dab4ad88a4eb9903cec1ff3d6c983" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 32967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 } health_report { overall_health: UNKNOWN } } }
I20260619 06:55:25.246168 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.240527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46176 (local address 127.18.39.65:32967)
0619 06:55:25.240729 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.240734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.240752 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:55:25.240945 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.240949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.241053 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.241179 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:25.241686 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.242489 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.243960 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.244383 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.244484 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.244946 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:25.245549 (+ 603us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:25.245668 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.245772 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.245842 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 06:55:25.247263 4166 tablet.cc:2369] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:25.254729 4120 raft_consensus.cc:1261] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 1 FOLLOWER]: Refusing update from remote peer 922dab4ad88a4eb9903cec1ff3d6c983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:25.254751 4250 raft_consensus.cc:1261] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 1 FOLLOWER]: Refusing update from remote peer 922dab4ad88a4eb9903cec1ff3d6c983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:25.255769 4324 consensus_queue.cc:1035] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4686d3cdd3a74f9e92769625fcd076f3" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 37757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:55:25.256147 4035 tablet.cc:2369] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:25.256196 4319 consensus_queue.cc:1035] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aca7b37813485880e105548283c319" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:25.282508 3848 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:25.282799 3848 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:25.286589 3848 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 739842fadebe47b590a8f3c213f0b1fb: Sending DeleteTablet for 3 replicas of tablet 1ad4361e7e8743549ce6f49f2cbd43ed
I20260619 06:55:25.287842 4230 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:25 UTC) from {username='slave'} at 127.0.0.1:45230
I20260619 06:55:25.287812 3969 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:25 UTC) from {username='slave'} at 127.0.0.1:46174
I20260619 06:55:25.287947 4100 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ad4361e7e8743549ce6f49f2cbd43ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:25 UTC) from {username='slave'} at 127.0.0.1:33710
I20260619 06:55:25.289966 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3907
I20260619 06:55:25.301659 4337 tablet_replica.cc:330] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: stopping tablet replica
I20260619 06:55:25.302322 4337 raft_consensus.cc:2229] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:25.302877 4337 raft_consensus.cc:2258] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:25.303958 4338 tablet_replica.cc:330] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983: stopping tablet replica
I20260619 06:55:25.304447 4338 raft_consensus.cc:2229] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:25.305945 4339 tablet_replica.cc:330] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: stopping tablet replica
I20260619 06:55:25.306392 4339 raft_consensus.cc:2229] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:25.306558 4337 ts_tablet_manager.cc:1905] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:25.306906 4339 raft_consensus.cc:2258] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:25.307229 4338 raft_consensus.cc:2258] T 1ad4361e7e8743549ce6f49f2cbd43ed P 922dab4ad88a4eb9903cec1ff3d6c983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:25.309064 4339 ts_tablet_manager.cc:1905] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:25.313634 4337 ts_tablet_manager.cc:1918] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:25.313818 4337 log.cc:1196] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/1ad4361e7e8743549ce6f49f2cbd43ed
I20260619 06:55:25.314337 4337 ts_tablet_manager.cc:1939] T 1ad4361e7e8743549ce6f49f2cbd43ed P 4686d3cdd3a74f9e92769625fcd076f3: Deleting consensus metadata
I20260619 06:55:25.316009 3834 catalog_manager.cc:4841] TS 4686d3cdd3a74f9e92769625fcd076f3 (127.18.39.67:37757): tablet 1ad4361e7e8743549ce6f49f2cbd43ed (table test-table [id=604dd88b485b470a8e9491f7cde8cb2b]) successfully deleted
I20260619 06:55:25.316195 4339 ts_tablet_manager.cc:1918] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:25.311563 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.291235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.18.39.65:32967)
0619 06:55:25.291484 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.291494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.298290 (+ 6796us) server_negotiation.cc:407] Connection header received
0619 06:55:25.298508 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.298518 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.298676 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.298800 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:25.303827 (+ 5027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.305255 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.307472 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.308143 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.308252 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.309818 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:25.309829 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:25.309846 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:25.310612 (+ 766us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:25.310671 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:25.310697 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:25.310783 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.311036 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.311168 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 06:55:25.316371 4339 log.cc:1196] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/1ad4361e7e8743549ce6f49f2cbd43ed
I20260619 06:55:25.316944 4339 ts_tablet_manager.cc:1939] T 1ad4361e7e8743549ce6f49f2cbd43ed P 59aca7b37813485880e105548283c319: Deleting consensus metadata
I20260619 06:55:25.317210 4029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:25.318784 3835 catalog_manager.cc:4841] TS 59aca7b37813485880e105548283c319 (127.18.39.66:43667): tablet 1ad4361e7e8743549ce6f49f2cbd43ed (table test-table [id=604dd88b485b470a8e9491f7cde8cb2b]) successfully deleted
I20260619 06:55:25.415437 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3907
W20260619 06:55:25.433720 3833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.39.65:32967 (error 108)
I20260619 06:55:25.434412 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4038
I20260619 06:55:25.435724 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.435273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46206)
0619 06:55:25.435459 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.435576 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 06:55:25.435946 3833 catalog_manager.cc:4584] TS 922dab4ad88a4eb9903cec1ff3d6c983 (127.18.39.65:32967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ad4361e7e8743549ce6f49f2cbd43ed: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
I20260619 06:55:25.442226 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.435372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.18.39.66:43667)
0619 06:55:25.435579 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.435588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.435610 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:55:25.435680 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.435691 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.435818 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.435927 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:25.436578 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.437475 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.438881 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.439306 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.439407 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.440593 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:25.440600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:25.440638 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:25.441217 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:25.441261 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:25.441284 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:25.441637 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.441770 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.441857 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:55:25.442943 4160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:25.489547 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.488981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46214)
0619 06:55:25.489206 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.489317 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 06:55:25.490581 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.490218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46228)
0619 06:55:25.490365 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.490446 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 06:55:25.541915 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4038
I20260619 06:55:25.551450 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.550855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46242)
0619 06:55:25.551106 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.551236 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260619 06:55:25.552696 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.552232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46258)
0619 06:55:25.552420 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.552517 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 06:55:25.559989 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4168
I20260619 06:55:25.566689 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.560869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45236 (local address 127.18.39.67:37757)
0619 06:55:25.561036 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:25.561040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:25.561059 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:55:25.561139 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:25.561143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:25.561232 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:25.561396 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:25.561787 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.562537 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.563762 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:25.564152 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:25.564248 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:25.565283 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:25.565292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:25.565306 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:25.566026 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:25.566098 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:25.566124 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:25.566230 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:25.566362 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 06:55:25.566447 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:55:25.567312 4290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:25.633620 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.632982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46272)
0619 06:55:25.633292 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.633443 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260619 06:55:25.634661 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:25.634269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:32967 (local address 127.0.0.1:46274)
0619 06:55:25.634473 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:25.634551 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 06:55:25.657323 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4168
I20260619 06:55:25.673676 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 3817
I20260619 06:55:25.674945 3878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:25.787410 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 3817
2026-06-19T06:55:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2873 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:25 dist-test-slave-1h5s krb5kdc[4348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:25 dist-test-slave-1h5s krb5kdc[4348](info): set up 2 sockets
Jun 19 06:55:25 dist-test-slave-1h5s krb5kdc[4348](info): commencing operation
krb5kdc: starting...
W20260619 06:55:26.913652 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:27 dist-test-slave-1h5s krb5kdc[4348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:55:27.163991 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35775
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:27.498952 4361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:27.499274 4361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:27.499444 4361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:27.506870 4361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:27.506995 4361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:27.507092 4361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:27.507182 4361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:27.507264 4361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:27.517376 4361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:27.518865 4361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:27.520202 4361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:27.532498 4366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:27.532480 4367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:27.532480 4369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:27.533154 4361 server_base.cc:1034] running on GCE node
I20260619 06:55:27.534049 4361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:27.535583 4361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:27.536798 4361 hybrid_clock.cc:648] HybridClock initialized: now 1781852127536756 us; error 33 us; skew 500 ppm
I20260619 06:55:27.537171 4361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:27 dist-test-slave-1h5s krb5kdc[4348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:27.545277 4361 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:55:27.547156 4361 webserver.cc:460] Webserver started at http://127.18.39.126:35863/ using document root <none> and password file <none>
I20260619 06:55:27.547914 4361 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:27.548063 4361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:27.548401 4361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:27.551664 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "59035cbef5bd45f383829f248a87e282"
format_stamp: "Formatted at 2026-06-19 06:55:27 on dist-test-slave-1h5s"
server_key: "8e58fc0e135ce96233bfd201007dc6d2"
server_key_iv: "95c0713d2d400a6100778377548aaeda"
server_key_version: "encryptionkey@0"
I20260619 06:55:27.552498 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "59035cbef5bd45f383829f248a87e282"
format_stamp: "Formatted at 2026-06-19 06:55:27 on dist-test-slave-1h5s"
server_key: "8e58fc0e135ce96233bfd201007dc6d2"
server_key_iv: "95c0713d2d400a6100778377548aaeda"
server_key_version: "encryptionkey@0"
I20260619 06:55:27.558281 4361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:55:27.562372 4376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:27.564054 4361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:55:27.564323 4361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "59035cbef5bd45f383829f248a87e282"
format_stamp: "Formatted at 2026-06-19 06:55:27 on dist-test-slave-1h5s"
server_key: "8e58fc0e135ce96233bfd201007dc6d2"
server_key_iv: "95c0713d2d400a6100778377548aaeda"
server_key_version: "encryptionkey@0"
I20260619 06:55:27.564566 4361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:27.592605 4361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:27.597079 4361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:27.597477 4361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:27.616843 4361 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35775
I20260619 06:55:27.616873 4427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35775 every 8 connection(s)
I20260619 06:55:27.618414 4361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:27.619457 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4361
I20260619 06:55:27.619716 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:27.620296 18589 external_mini_cluster.cc:1428] Setting key a472d6243976c3481995f82b2a57ecf8
I20260619 06:55:27.622975 4428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:55:27 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:27.637497 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: Bootstrap starting.
I20260619 06:55:27.641803 4428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:27.643124 4428 log.cc:824] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:27.646284 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: No bootstrap required, opened a new log
I20260619 06:55:27.646919 4431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:27.622646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.18.39.126:35775)
0619 06:55:27.623662 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:27.623678 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:27.624080 (+ 402us) server_negotiation.cc:407] Connection header received
0619 06:55:27.626304 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:27.626333 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:27.626907 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:27.627390 (+ 483us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:27.628807 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:27.630928 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:27.632624 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:27.633750 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:27.633957 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:27.638453 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:27.638463 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:27.638482 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:27.638531 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:27.642588 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:27.643428 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:27.643436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:27.643442 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:27.643628 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:27.644050 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:27.644067 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:27.644070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:27.644637 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:27.644936 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:27.645423 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 06:55:27.645864 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":439,"threads_started":1}
I20260619 06:55:27.654848 4428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } }
I20260619 06:55:27.655285 4428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:27.655395 4428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59035cbef5bd45f383829f248a87e282, State: Initialized, Role: FOLLOWER
I20260619 06:55:27.656035 4428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } }
I20260619 06:55:27.656281 4428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:27.656421 4428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:27.656608 4428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:27.659112 4428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } }
I20260619 06:55:27.659754 4428 leader_election.cc:302] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59035cbef5bd45f383829f248a87e282; no voters:
I20260619 06:55:27.660408 4428 leader_election.cc:288] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:27.660548 4433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:27.661432 4433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [term 1 LEADER]: Becoming Leader. State: Replica: 59035cbef5bd45f383829f248a87e282, State: Running, Role: LEADER
I20260619 06:55:27.662192 4433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } }
I20260619 06:55:27.662721 4428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:27.664973 4434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59035cbef5bd45f383829f248a87e282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } } }
I20260619 06:55:27.665551 4434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:27.665174 4435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59035cbef5bd45f383829f248a87e282. Latest consensus state: current_term: 1 leader_uuid: "59035cbef5bd45f383829f248a87e282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59035cbef5bd45f383829f248a87e282" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35775 } } }
I20260619 06:55:27.665733 4435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:27.670457 4439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:27.676501 4439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:27.685408 4439 catalog_manager.cc:1347] Generated new cluster ID: c2410d4560584520a7d2c86698d5ae16
I20260619 06:55:27.685580 4439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:27.719617 4439 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:27.721606 4439 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:27.742651 4439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: Generated new TSK 0
I20260619 06:55:27.743760 4439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:55:27.814325 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:28.154132 4454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:28.154426 4454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:28.154520 4454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:28.162045 4454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:28.162168 4454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:28.162285 4454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:28.172915 4454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:28.174283 4454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:28.175591 4454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:28.189081 4459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:28.189164 4462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:28.189141 4460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:28.190106 4454 server_base.cc:1034] running on GCE node
I20260619 06:55:28.191016 4454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:28.191987 4454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:28.193221 4454 hybrid_clock.cc:648] HybridClock initialized: now 1781852128193152 us; error 60 us; skew 500 ppm
I20260619 06:55:28.193640 4454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:28 dist-test-slave-1h5s krb5kdc[4348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:28.196833 4454 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:55:28.198426 4454 webserver.cc:460] Webserver started at http://127.18.39.65:45637/ using document root <none> and password file <none>
I20260619 06:55:28.199095 4454 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:28.199196 4454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:28.199448 4454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:28.202399 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "3648993f764b8b9d791a99947ba363f5"
server_key_iv: "80cff7735ff61e0a1bae10da4e863fa1"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.203104 4454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "3648993f764b8b9d791a99947ba363f5"
server_key_iv: "80cff7735ff61e0a1bae10da4e863fa1"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.208321 4454 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260619 06:55:28.211786 4469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.213153 4454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:55:28.213284 4454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "3648993f764b8b9d791a99947ba363f5"
server_key_iv: "80cff7735ff61e0a1bae10da4e863fa1"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.213573 4454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:28.252087 4454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:28.256418 4454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:28.256697 4454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:28.257718 4454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:28.259089 4454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:28.259169 4454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.259253 4454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:28.259297 4454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.293907 4454 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:40923
I20260619 06:55:28.293965 4581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:40923 every 8 connection(s)
I20260619 06:55:28.295264 4454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:28.299453 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4454
I20260619 06:55:28.299660 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:28.300263 18589 external_mini_cluster.cc:1428] Setting key 1c62b3155c61a1b75330b3be518949df
Jun 19 06:55:28 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:28.319274 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:28.297947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:46227 (local address 127.18.39.126:35775)
0619 06:55:28.298412 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:28.298417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:28.299539 (+ 1122us) server_negotiation.cc:407] Connection header received
0619 06:55:28.301950 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:28.301955 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:28.302020 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:28.302172 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:28.303832 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.305030 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.306717 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.307436 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.307601 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:28.311595 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:28.311605 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:28.311613 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:28.311662 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:28.315376 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:28.316282 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:28.316290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:28.316296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:28.316499 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:28.317012 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:28.317019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:28.317024 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:28.317448 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:28.317612 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:28.318496 (+ 884us) server_negotiation.cc:299] Negotiation successful
0619 06:55:28.318787 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":231,"threads_started":1}
I20260619 06:55:28.319943 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:28.298545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35775 (local address 127.18.39.65:46227)
0619 06:55:28.299341 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:28.299390 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:28.301638 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:28.302481 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:28.302494 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:28.303050 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:28.303640 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:28.303663 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.305199 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.305203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:28.306540 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:28.306559 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.307730 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.307736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:28.307963 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:28.308748 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:28.308772 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:28.311291 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:28.315569 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:28.315578 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:28.315594 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:28.316115 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:28.316672 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:28.316676 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:28.316680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:28.316893 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:28.317582 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:28.317590 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:28.318035 (+ 445us) client_negotiation.cc:769] Sending connection context
0619 06:55:28.318328 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 06:55:28.318742 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":269,"threads_started":1}
I20260619 06:55:28.321751 4582 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35775
I20260619 06:55:28.322117 4582 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:28.323083 4582 heartbeater.cc:511] Master 127.18.39.126:35775 requested a full tablet report, sending...
I20260619 06:55:28.325608 4393 ts_manager.cc:194] Registered new tserver with Master: 20f2b6c053d54b228b9c24f7f2e5b7d4 (127.18.39.65:40923)
I20260619 06:55:28.327086 4393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:46227
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:55:28.358982 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:28.688683 4588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:28.688988 4588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:28.689141 4588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:28.696609 4588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:28.696733 4588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:28.696919 4588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:28.707383 4588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:28.709375 4588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:28.711032 4588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:28.724905 4594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:28.724946 4593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:28.724875 4596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:28.725405 4588 server_base.cc:1034] running on GCE node
I20260619 06:55:28.726303 4588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:28.727243 4588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:28.728524 4588 hybrid_clock.cc:648] HybridClock initialized: now 1781852128728436 us; error 57 us; skew 500 ppm
I20260619 06:55:28.728920 4588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:28 dist-test-slave-1h5s krb5kdc[4348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:28.732300 4588 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:55:28.733987 4588 webserver.cc:460] Webserver started at http://127.18.39.66:39827/ using document root <none> and password file <none>
I20260619 06:55:28.734665 4588 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:28.734786 4588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:28.735052 4588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:28.738183 4588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "815100f749854beea05d5caf0f27869d"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "ca769a9b055eaeed3d902a9139cd538f"
server_key_iv: "832a6e1aa581d085a01768c0c7b8ffeb"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.738947 4588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "815100f749854beea05d5caf0f27869d"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "ca769a9b055eaeed3d902a9139cd538f"
server_key_iv: "832a6e1aa581d085a01768c0c7b8ffeb"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.744374 4588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:55:28.747934 4603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.749280 4588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:28.749491 4588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "815100f749854beea05d5caf0f27869d"
format_stamp: "Formatted at 2026-06-19 06:55:28 on dist-test-slave-1h5s"
server_key: "ca769a9b055eaeed3d902a9139cd538f"
server_key_iv: "832a6e1aa581d085a01768c0c7b8ffeb"
server_key_version: "encryptionkey@0"
I20260619 06:55:28.749764 4588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:28.782565 4588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:28.787540 4588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:28.787896 4588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:28.788965 4588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:28.790419 4588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:28.790494 4588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.790704 4588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:28.790766 4588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:28.824064 4588 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:33487
I20260619 06:55:28.824128 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:33487 every 8 connection(s)
I20260619 06:55:28.825588 4588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:28.834614 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4588
I20260619 06:55:28.834805 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:28.835238 18589 external_mini_cluster.cc:1428] Setting key e05cb0b12f7484c717ba00bb13e779a5
Jun 19 06:55:28 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852128, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:28.849677 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:28.828128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:53695 (local address 127.18.39.126:35775)
0619 06:55:28.828588 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:28.828595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:28.829565 (+ 970us) server_negotiation.cc:407] Connection header received
0619 06:55:28.831420 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:28.831425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:28.831499 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:28.831607 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:28.833185 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.833969 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.835719 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.836340 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.836500 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:28.841998 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:28.842006 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:28.842011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:28.842062 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:28.845891 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:28.846966 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:28.846970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:28.846972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:28.847098 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:28.847597 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:28.847600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:28.847602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:28.847867 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:28.847978 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:28.848870 (+ 892us) server_negotiation.cc:299] Negotiation successful
0619 06:55:28.849136 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":209,"threads_started":1}
I20260619 06:55:28.850574 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:28.828670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35775 (local address 127.18.39.66:53695)
0619 06:55:28.829386 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:28.829438 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:28.831136 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:28.831857 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:28.831865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:28.832394 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:28.832994 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:28.833011 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.834152 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.834156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:28.835563 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:28.835575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:28.836803 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:28.836809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:28.837109 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:28.838168 (+ 1059us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:28.838216 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:28.841662 (+ 3446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:28.846070 (+ 4408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:28.846084 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:28.846114 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:28.846830 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:28.847209 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:28.847216 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:28.847222 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:28.847487 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:28.847998 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:28.848010 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:28.848489 (+ 479us) client_negotiation.cc:769] Sending connection context
0619 06:55:28.848733 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 06:55:28.849158 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
I20260619 06:55:28.852270 4716 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35775
I20260619 06:55:28.852622 4716 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:28.853509 4716 heartbeater.cc:511] Master 127.18.39.126:35775 requested a full tablet report, sending...
I20260619 06:55:28.855185 4393 ts_manager.cc:194] Registered new tserver with Master: 815100f749854beea05d5caf0f27869d (127.18.39.66:33487)
I20260619 06:55:28.856503 4393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:53695
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:55:28.894933 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:29.224627 4722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:29.224925 4722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:29.225088 4722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:29.232527 4722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:29.232713 4722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:29.232920 4722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:29.243213 4722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:29.244575 4722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:29.245967 4722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:29.259469 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:29.259537 4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:29.259606 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:29.260275 4722 server_base.cc:1034] running on GCE node
I20260619 06:55:29.261090 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:29.262163 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:29.263433 4722 hybrid_clock.cc:648] HybridClock initialized: now 1781852129263373 us; error 39 us; skew 500 ppm
I20260619 06:55:29.263803 4722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:29 dist-test-slave-1h5s krb5kdc[4348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852129, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:29.267180 4722 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:55:29.268874 4722 webserver.cc:460] Webserver started at http://127.18.39.67:33943/ using document root <none> and password file <none>
I20260619 06:55:29.269735 4722 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:29.269868 4722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:29.270226 4722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:29.273479 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "baadc5780c524b3c89eae09e0328c43f"
format_stamp: "Formatted at 2026-06-19 06:55:29 on dist-test-slave-1h5s"
server_key: "5f5d0a3468ca84540031640b4bd62875"
server_key_iv: "f9492ac21b7091caae965b7af0926235"
server_key_version: "encryptionkey@0"
I20260619 06:55:29.274322 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "baadc5780c524b3c89eae09e0328c43f"
format_stamp: "Formatted at 2026-06-19 06:55:29 on dist-test-slave-1h5s"
server_key: "5f5d0a3468ca84540031640b4bd62875"
server_key_iv: "f9492ac21b7091caae965b7af0926235"
server_key_version: "encryptionkey@0"
I20260619 06:55:29.279768 4722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:29.283432 4737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:29.284881 4722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:29.285085 4722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "baadc5780c524b3c89eae09e0328c43f"
format_stamp: "Formatted at 2026-06-19 06:55:29 on dist-test-slave-1h5s"
server_key: "5f5d0a3468ca84540031640b4bd62875"
server_key_iv: "f9492ac21b7091caae965b7af0926235"
server_key_version: "encryptionkey@0"
I20260619 06:55:29.285308 4722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:29.306921 4722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:29.311108 4722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:29.311411 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:29.312415 4722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:29.313767 4722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:29.313879 4722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:29.314025 4722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:29.314100 4722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:29.330799 4582 heartbeater.cc:503] Master 127.18.39.126:35775 was elected leader, sending a full tablet report...
I20260619 06:55:29.348167 4722 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:43407
I20260619 06:55:29.348263 4849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:43407 every 8 connection(s)
I20260619 06:55:29.349619 4722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:29.359450 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4722
I20260619 06:55:29.359647 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:29.360148 18589 external_mini_cluster.cc:1428] Setting key 7577201e42e0ae7e2a1b4e2161fc025f
Jun 19 06:55:29 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852129, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:29.371845 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.352347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:45613 (local address 127.18.39.126:35775)
0619 06:55:29.352863 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.352868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.354025 (+ 1157us) server_negotiation.cc:407] Connection header received
0619 06:55:29.356107 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.356114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.356211 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.356353 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:29.358095 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.358928 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.361005 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.361699 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.361866 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.365846 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:29.365851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:29.365854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:29.365887 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:29.368545 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.369410 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.369420 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.369423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.369542 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.369986 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.369997 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.370000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.370282 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:29.370443 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.371220 (+ 777us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.371446 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":269,"threads_started":1}
I20260619 06:55:29.372787 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.352896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35775 (local address 127.18.39.67:45613)
0619 06:55:29.353847 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.353897 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.355777 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.356616 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.356625 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.357197 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:29.357887 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.357905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.359146 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.359151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.360846 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.360869 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.361965 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.361973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.362295 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.363173 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:29.363220 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:29.365593 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:29.368697 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:29.368706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:29.368724 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:29.369258 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:29.369661 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:29.369666 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:29.369670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:29.369868 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:29.370415 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:29.370423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:29.370791 (+ 368us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.371067 (+ 276us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.371631 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":268,"threads_started":1}
I20260619 06:55:29.374444 4850 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35775
I20260619 06:55:29.374787 4850 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:29.375694 4850 heartbeater.cc:511] Master 127.18.39.126:35775 requested a full tablet report, sending...
I20260619 06:55:29.377048 4393 ts_manager.cc:194] Registered new tserver with Master: baadc5780c524b3c89eae09e0328c43f (127.18.39.67:43407)
I20260619 06:55:29.377928 4393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:45613
I20260619 06:55:29.388299 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:29.405732 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.392704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.18.39.126:35775)
0619 06:55:29.392940 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.392950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.393287 (+ 337us) server_negotiation.cc:407] Connection header received
0619 06:55:29.393494 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.393498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.393595 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.393683 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:29.395170 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.395897 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.397288 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.397714 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.397814 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.400575 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:29.400579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:29.400583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:29.400613 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:29.403131 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.403914 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.403918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.403920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.404029 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.404391 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.404394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.404396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.404683 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:29.404884 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.405187 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.405322 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:55:29.409785 4393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:29.416847 4393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:29.443091 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.429287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.18.39.66:33487)
0619 06:55:29.429930 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.429938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.430609 (+ 671us) server_negotiation.cc:407] Connection header received
0619 06:55:29.431057 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.431067 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.431336 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.431538 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:29.432848 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.434993 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.439498 (+ 4505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.441390 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.441506 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.441727 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.442537 (+ 810us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.442715 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":1189,"threads_started":1}
I20260619 06:55:29.443964 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.429080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:33487 (local address 127.0.0.1:38608)
0619 06:55:29.430462 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.430504 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.430777 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.431947 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.431958 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.431999 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:29.432582 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.432599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.435251 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.435260 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.439245 (+ 3985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.439269 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.441942 (+ 2673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.441954 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.442231 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.442308 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.443388 (+ 1080us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.443602 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"thread_start_us":12251,"threads_started":1}
I20260619 06:55:29.447925 4651 tablet_service.cc:1467] Processing CreateTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a (DEFAULT_TABLE table=test-table [id=75b3f33551c1477d9c2081d7c009a1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:29.449473 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.429713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40923 (local address 127.0.0.1:34590)
0619 06:55:29.442195 (+ 12482us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.442213 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.442324 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.442957 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.442961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.442991 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:29.443331 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.443345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.445030 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.445037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.447286 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.447295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.448596 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.448602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.448784 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.448829 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.448939 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.449091 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12361,"thread_start_us":992,"threads_started":1}
I20260619 06:55:29.449620 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6502b74fa610451e8fbcc4ff6424db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:29.449721 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.430343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.18.39.65:40923)
0619 06:55:29.441699 (+ 11356us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.441707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.442450 (+ 743us) server_negotiation.cc:407] Connection header received
0619 06:55:29.442565 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.442573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.442784 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.442976 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:29.443518 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.444849 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.447432 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.448381 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.448550 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.448788 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.449139 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.449305 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10874,"thread_start_us":11085,"threads_started":1}
I20260619 06:55:29.450106 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.430121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.18.39.67:43407)
0619 06:55:29.440309 (+ 10188us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.440321 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.441417 (+ 1096us) server_negotiation.cc:407] Connection header received
0619 06:55:29.441780 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.441792 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.442115 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.442342 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:29.442909 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.444539 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.447267 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.448646 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.448781 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.449010 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.449580 (+ 570us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.449769 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9975,"thread_start_us":9775,"threads_started":1}
I20260619 06:55:29.450721 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.429409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43407 (local address 127.0.0.1:60294)
0619 06:55:29.441163 (+ 11754us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.441183 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.441318 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.442321 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.442326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.442353 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:29.442745 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.442755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.444679 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.444684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.447076 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.447092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.448947 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.448953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.449119 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.449156 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.449246 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.449390 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11626,"thread_start_us":12509,"threads_started":1}
I20260619 06:55:29.454526 4517 tablet_service.cc:1467] Processing CreateTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a (DEFAULT_TABLE table=test-table [id=75b3f33551c1477d9c2081d7c009a1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:29.454526 4785 tablet_service.cc:1467] Processing CreateTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a (DEFAULT_TABLE table=test-table [id=75b3f33551c1477d9c2081d7c009a1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:29.456074 4785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6502b74fa610451e8fbcc4ff6424db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:29.456074 4517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6502b74fa610451e8fbcc4ff6424db0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:29.462419 4869 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Bootstrap starting.
I20260619 06:55:29.466415 4869 tablet_bootstrap.cc:654] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:29.467506 4869 log.cc:824] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:29.470522 4869 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: No bootstrap required, opened a new log
I20260619 06:55:29.470829 4869 ts_tablet_manager.cc:1397] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 06:55:29.478169 4871 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Bootstrap starting.
I20260619 06:55:29.479025 4872 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Bootstrap starting.
I20260619 06:55:29.479357 4869 raft_consensus.cc:348] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.479818 4869 raft_consensus.cc:374] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:29.479916 4869 raft_consensus.cc:729] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815100f749854beea05d5caf0f27869d, State: Initialized, Role: FOLLOWER
I20260619 06:55:29.480552 4869 consensus_queue.cc:260] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.481753 4716 heartbeater.cc:503] Master 127.18.39.126:35775 was elected leader, sending a full tablet report...
I20260619 06:55:29.482388 4869 ts_tablet_manager.cc:1428] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 06:55:29.482887 4872 tablet_bootstrap.cc:654] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:29.484229 4872 log.cc:824] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:29.484885 4871 tablet_bootstrap.cc:654] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:29.486042 4871 log.cc:824] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:29.489032 4871 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: No bootstrap required, opened a new log
I20260619 06:55:29.489048 4872 tablet_bootstrap.cc:492] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: No bootstrap required, opened a new log
I20260619 06:55:29.489377 4872 ts_tablet_manager.cc:1397] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260619 06:55:29.489378 4871 ts_tablet_manager.cc:1397] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.005s
I20260619 06:55:29.500736 4871 raft_consensus.cc:348] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.500736 4872 raft_consensus.cc:348] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.501269 4871 raft_consensus.cc:374] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:29.501271 4872 raft_consensus.cc:374] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:29.501406 4871 raft_consensus.cc:729] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baadc5780c524b3c89eae09e0328c43f, State: Initialized, Role: FOLLOWER
I20260619 06:55:29.501402 4872 raft_consensus.cc:729] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f2b6c053d54b228b9c24f7f2e5b7d4, State: Initialized, Role: FOLLOWER
I20260619 06:55:29.502156 4872 consensus_queue.cc:260] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.502148 4871 consensus_queue.cc:260] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.503816 4850 heartbeater.cc:503] Master 127.18.39.126:35775 was elected leader, sending a full tablet report...
I20260619 06:55:29.504287 4872 ts_tablet_manager.cc:1428] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260619 06:55:29.504405 4871 ts_tablet_manager.cc:1428] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
W20260619 06:55:29.547871 4583 tablet.cc:2369] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:29.553896 4873 raft_consensus.cc:482] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:29.554284 4873 raft_consensus.cc:504] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.555888 4873 leader_election.cc:288] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baadc5780c524b3c89eae09e0328c43f (127.18.39.67:43407), 20f2b6c053d54b228b9c24f7f2e5b7d4 (127.18.39.65:40923)
I20260619 06:55:29.565788 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.556467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:56951 (local address 127.18.39.65:40923)
0619 06:55:29.556796 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.556801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.557471 (+ 670us) server_negotiation.cc:407] Connection header received
0619 06:55:29.557669 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.557673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.557731 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.557828 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:29.558980 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.560048 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.563175 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.564467 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.564639 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.564720 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.565376 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.565524 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260619 06:55:29.565809 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.556340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:43267 (local address 127.18.39.67:43407)
0619 06:55:29.556575 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.556584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.557359 (+ 775us) server_negotiation.cc:407] Connection header received
0619 06:55:29.557592 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.557596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.557687 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.557793 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:29.558834 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.560021 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.563168 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.564470 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.564676 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.564746 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.565383 (+ 637us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.565527 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:55:29.566118 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.556292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40923 (local address 127.18.39.66:56951)
0619 06:55:29.557314 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.557375 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.557499 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.557840 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.557844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.558230 (+ 386us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:29.558746 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.558760 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.560203 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.560208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.562988 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.562998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.564790 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.564795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.565043 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.565197 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.565296 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.565535 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":392,"spinlock_wait_cycles":31104,"thread_start_us":439,"threads_started":1}
I20260619 06:55:29.566272 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.556207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43407 (local address 127.18.39.66:43267)
0619 06:55:29.557213 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:29.557230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:29.557454 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:29.557809 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:29.557814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:29.557836 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:29.558613 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.558632 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.560202 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.560208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.562974 (+ 2766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:29.562991 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.564787 (+ 1796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.564794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:29.565043 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.565197 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 06:55:29.565296 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:55:29.565535 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":247,"spinlock_wait_cycles":596736,"thread_start_us":759,"threads_started":1}
I20260619 06:55:29.567217 4537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6502b74fa610451e8fbcc4ff6424db0a" candidate_uuid: "815100f749854beea05d5caf0f27869d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" is_pre_election: true
I20260619 06:55:29.567274 4805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6502b74fa610451e8fbcc4ff6424db0a" candidate_uuid: "815100f749854beea05d5caf0f27869d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baadc5780c524b3c89eae09e0328c43f" is_pre_election: true
I20260619 06:55:29.567747 4537 raft_consensus.cc:2454] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815100f749854beea05d5caf0f27869d in term 0.
I20260619 06:55:29.567802 4805 raft_consensus.cc:2454] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 815100f749854beea05d5caf0f27869d in term 0.
I20260619 06:55:29.568480 4604 leader_election.cc:302] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20f2b6c053d54b228b9c24f7f2e5b7d4, 815100f749854beea05d5caf0f27869d; no voters:
I20260619 06:55:29.568996 4873 raft_consensus.cc:2781] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:29.569155 4873 raft_consensus.cc:482] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:29.569289 4873 raft_consensus.cc:3037] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:29.571689 4873 raft_consensus.cc:504] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.572386 4873 leader_election.cc:288] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [CANDIDATE]: Term 1 election: Requested vote from peers baadc5780c524b3c89eae09e0328c43f (127.18.39.67:43407), 20f2b6c053d54b228b9c24f7f2e5b7d4 (127.18.39.65:40923)
I20260619 06:55:29.572903 4805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6502b74fa610451e8fbcc4ff6424db0a" candidate_uuid: "815100f749854beea05d5caf0f27869d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baadc5780c524b3c89eae09e0328c43f"
I20260619 06:55:29.572947 4537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6502b74fa610451e8fbcc4ff6424db0a" candidate_uuid: "815100f749854beea05d5caf0f27869d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4"
I20260619 06:55:29.573148 4805 raft_consensus.cc:3037] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:29.573143 4537 raft_consensus.cc:3037] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:29.576413 4805 raft_consensus.cc:2454] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815100f749854beea05d5caf0f27869d in term 1.
I20260619 06:55:29.576413 4537 raft_consensus.cc:2454] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 815100f749854beea05d5caf0f27869d in term 1.
I20260619 06:55:29.576941 4607 leader_election.cc:302] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20f2b6c053d54b228b9c24f7f2e5b7d4, 815100f749854beea05d5caf0f27869d, baadc5780c524b3c89eae09e0328c43f; no voters:
I20260619 06:55:29.577347 4873 raft_consensus.cc:2781] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 1 FOLLOWER]: Leader election won for term 1
W20260619 06:55:29.577667 4717 tablet.cc:2369] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:29.577932 4873 raft_consensus.cc:686] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 1 LEADER]: Becoming Leader. State: Replica: 815100f749854beea05d5caf0f27869d, State: Running, Role: LEADER
I20260619 06:55:29.578475 4873 consensus_queue.cc:237] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } }
I20260619 06:55:29.582973 4393 catalog_manager.cc:5495] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d reported cstate change: term changed from 0 to 1, leader changed from <none> to 815100f749854beea05d5caf0f27869d (127.18.39.66). New cstate: current_term: 1 leader_uuid: "815100f749854beea05d5caf0f27869d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "815100f749854beea05d5caf0f27869d" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 } health_report { overall_health: UNKNOWN } } }
W20260619 06:55:29.601620 4851 tablet.cc:2369] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:29.602088 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.596284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.18.39.66:33487)
0619 06:55:29.596509 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.596516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.596626 (+ 110us) server_negotiation.cc:407] Connection header received
0619 06:55:29.596892 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.596896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.596974 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.597127 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:29.597612 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.598354 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.599803 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.600228 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.600367 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.600912 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:29.601487 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:29.601678 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.601790 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.601835 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:55:29.610203 4537 raft_consensus.cc:1261] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 1 FOLLOWER]: Refusing update from remote peer 815100f749854beea05d5caf0f27869d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:29.610206 4805 raft_consensus.cc:1261] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 1 FOLLOWER]: Refusing update from remote peer 815100f749854beea05d5caf0f27869d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:29.611219 4880 consensus_queue.cc:1035] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [LEADER]: Connected to new peer: Peer: permanent_uuid: "20f2b6c053d54b228b9c24f7f2e5b7d4" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:29.611562 4873 consensus_queue.cc:1035] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [LEADER]: Connected to new peer: Peer: permanent_uuid: "baadc5780c524b3c89eae09e0328c43f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:29.631690 4884 mvcc.cc:205] Tried to move back new op lower bound from 7298466322873143296 to 7298466322757111808. Current Snapshot: MvccSnapshot[applied={T|T < 7298466322873143296}]
I20260619 06:55:29.647233 4393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:29.647606 4393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:29.652655 4393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59035cbef5bd45f383829f248a87e282: Sending DeleteTablet for 3 replicas of tablet 6502b74fa610451e8fbcc4ff6424db0a
I20260619 06:55:29.654006 4651 tablet_service.cc:1514] Processing DeleteTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:29 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38608
I20260619 06:55:29.654492 4785 tablet_service.cc:1514] Processing DeleteTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:29 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60294
I20260619 06:55:29.656018 4784 tablet_service.cc:1514] Processing DeleteTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:29 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:60294
I20260619 06:55:29.657315 4517 tablet_service.cc:1514] Processing DeleteTablet for tablet 6502b74fa610451e8fbcc4ff6424db0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:29 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:34590
I20260619 06:55:29.663009 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4454
I20260619 06:55:29.665091 4894 tablet_replica.cc:330] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: stopping tablet replica
I20260619 06:55:29.665601 4894 raft_consensus.cc:2229] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:29.666162 4894 raft_consensus.cc:2258] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:29.666975 4893 tablet_replica.cc:330] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: stopping tablet replica
I20260619 06:55:29.667371 4893 raft_consensus.cc:2229] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:29.668058 4893 raft_consensus.cc:2258] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:29.669159 4894 ts_tablet_manager.cc:1905] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:29.670552 4895 tablet_replica.cc:330] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: stopping tablet replica
I20260619 06:55:29.671056 4895 raft_consensus.cc:2229] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:29.671617 4895 raft_consensus.cc:2258] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:29.673434 4893 ts_tablet_manager.cc:1905] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:29.676780 4894 ts_tablet_manager.cc:1918] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:29.677037 4894 log.cc:1196] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/6502b74fa610451e8fbcc4ff6424db0a
I20260619 06:55:29.677789 4894 ts_tablet_manager.cc:1939] T 6502b74fa610451e8fbcc4ff6424db0a P baadc5780c524b3c89eae09e0328c43f: Deleting consensus metadata
I20260619 06:55:29.679929 4380 catalog_manager.cc:4841] TS baadc5780c524b3c89eae09e0328c43f (127.18.39.67:43407): tablet 6502b74fa610451e8fbcc4ff6424db0a (table test-table [id=75b3f33551c1477d9c2081d7c009a1e9]) successfully deleted
Jun 19 06:55:29 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:55:29.680908 4893 ts_tablet_manager.cc:1918] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:29.681138 4893 log.cc:1196] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/6502b74fa610451e8fbcc4ff6424db0a
W20260619 06:55:29.681146 4380 catalog_manager.cc:4808] TS baadc5780c524b3c89eae09e0328c43f (127.18.39.67:43407): delete failed for tablet 6502b74fa610451e8fbcc4ff6424db0a because the tablet was not found. No further retry: Not found: Tablet not found: 6502b74fa610451e8fbcc4ff6424db0a
I20260619 06:55:29.681739 4893 ts_tablet_manager.cc:1939] T 6502b74fa610451e8fbcc4ff6424db0a P 815100f749854beea05d5caf0f27869d: Deleting consensus metadata
I20260619 06:55:29.682967 4895 ts_tablet_manager.cc:1905] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:29.683442 4379 catalog_manager.cc:4841] TS 815100f749854beea05d5caf0f27869d (127.18.39.66:33487): tablet 6502b74fa610451e8fbcc4ff6424db0a (table test-table [id=75b3f33551c1477d9c2081d7c009a1e9]) successfully deleted
I20260619 06:55:29.690210 4895 ts_tablet_manager.cc:1918] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:29.690403 4895 log.cc:1196] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/6502b74fa610451e8fbcc4ff6424db0a
I20260619 06:55:29.690963 4895 ts_tablet_manager.cc:1939] T 6502b74fa610451e8fbcc4ff6424db0a P 20f2b6c053d54b228b9c24f7f2e5b7d4: Deleting consensus metadata
I20260619 06:55:29.691946 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.664818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.18.39.65:40923)
0619 06:55:29.665056 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.665065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.669870 (+ 4805us) server_negotiation.cc:407] Connection header received
0619 06:55:29.670047 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.670052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.670151 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.670256 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:29.672331 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.673483 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.675465 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.676130 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.676229 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.684203 (+ 7974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:29.684219 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:29.684236 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:29.684293 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:29.688431 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.689463 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.689473 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.689477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.689641 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.690119 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.690126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.690129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.690672 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:29.691003 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.691323 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.691558 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:55:29.692790 4577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:29.693213 4377 catalog_manager.cc:4841] TS 20f2b6c053d54b228b9c24f7f2e5b7d4 (127.18.39.65:40923): tablet 6502b74fa610451e8fbcc4ff6424db0a (table test-table [id=75b3f33551c1477d9c2081d7c009a1e9]) successfully deleted
I20260619 06:55:29.786357 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4454
I20260619 06:55:29.805897 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4588
Jun 19 06:55:29 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:55:29.832347 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.806654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.18.39.66:33487)
0619 06:55:29.806837 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.806842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.806909 (+ 67us) server_negotiation.cc:407] Connection header received
0619 06:55:29.807113 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.807116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.807181 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.807309 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:29.808902 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.810187 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.812360 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.813171 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.813286 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.821850 (+ 8564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:29.821873 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:29.821894 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:29.821949 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:29.828058 (+ 6109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.829154 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.829166 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.829170 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.829392 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.829884 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.829891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.829895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.830462 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:29.830804 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.831517 (+ 713us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.831789 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:55:29.833846 4711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:29.951584 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4588
I20260619 06:55:29.967734 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4722
Jun 19 06:55:29 dist-test-slave-1h5s krb5kdc[4348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:55:29.984408 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:29.968564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60300 (local address 127.18.39.67:43407)
0619 06:55:29.968784 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:29.968789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:29.968874 (+ 85us) server_negotiation.cc:407] Connection header received
0619 06:55:29.969015 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:29.969019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:29.969127 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:29.969214 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:29.970819 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.971759 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.973170 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:29.973697 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:29.973809 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:29.978328 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:29.978354 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:29.978371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:29.978408 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:29.981226 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.982401 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.982409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.982411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.982556 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:29.982943 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:29.982946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:29.982948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:29.983315 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:29.983525 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:29.983883 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 06:55:29.984115 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:55:29.985117 4845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:30.076550 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4722
I20260619 06:55:30.094448 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4361
I20260619 06:55:30.095541 4423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:30.207037 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4361
2026-06-19T06:55:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4417 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T06:55:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:30Z Disabled control of system clock
I20260619 06:55:30.283854 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35829
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:30.622969 4904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:30.623288 4904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:30.623369 4904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:30.630694 4904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:30.630817 4904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:30.630856 4904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:30.630908 4904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:30.630946 4904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:30.640439 4904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:30.641754 4904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:30.643448 4904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:30.655874 4910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:30.655903 4912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:30.655938 4909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:30.656728 4904 server_base.cc:1034] running on GCE node
I20260619 06:55:30.657734 4904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:30.659192 4904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:30.660398 4904 hybrid_clock.cc:648] HybridClock initialized: now 1781852130660356 us; error 32 us; skew 500 ppm
I20260619 06:55:30.660836 4904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:30.663374 4904 webserver.cc:460] Webserver started at http://127.18.39.126:46291/ using document root <none> and password file <none>
I20260619 06:55:30.664119 4904 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:30.664211 4904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:30.664557 4904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:30.667709 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "631cd097a0bd41d0aebf72278b2e6302"
format_stamp: "Formatted at 2026-06-19 06:55:30 on dist-test-slave-1h5s"
server_key: "3d33904cfff17725616094cc2a12d74a"
server_key_iv: "dd27b1e7061214e8fa43f5767bed88cb"
server_key_version: "encryptionkey@0"
I20260619 06:55:30.668536 4904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "631cd097a0bd41d0aebf72278b2e6302"
format_stamp: "Formatted at 2026-06-19 06:55:30 on dist-test-slave-1h5s"
server_key: "3d33904cfff17725616094cc2a12d74a"
server_key_iv: "dd27b1e7061214e8fa43f5767bed88cb"
server_key_version: "encryptionkey@0"
I20260619 06:55:30.673957 4904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:55:30.677670 4918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:30.679126 4904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:30.679337 4904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "631cd097a0bd41d0aebf72278b2e6302"
format_stamp: "Formatted at 2026-06-19 06:55:30 on dist-test-slave-1h5s"
server_key: "3d33904cfff17725616094cc2a12d74a"
server_key_iv: "dd27b1e7061214e8fa43f5767bed88cb"
server_key_version: "encryptionkey@0"
I20260619 06:55:30.679589 4904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:30.694833 4904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:30.695739 4904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:30.696048 4904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:30.717089 4904 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35829
I20260619 06:55:30.717106 4969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35829 every 8 connection(s)
I20260619 06:55:30.718840 4904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:30.722599 4970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:30.728754 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4904
I20260619 06:55:30.728971 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:30.729526 18589 external_mini_cluster.cc:1428] Setting key 1719ba66d5db5d0f4b4abee60038fd60
I20260619 06:55:30.733865 4970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: Bootstrap starting.
I20260619 06:55:30.737283 4970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:30.738596 4970 log.cc:824] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:30.742044 4970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: No bootstrap required, opened a new log
I20260619 06:55:30.744356 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:30.731678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53696 (local address 127.18.39.126:35829)
0619 06:55:30.732699 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:30.732711 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:30.732754 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:55:30.734594 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:30.734616 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:30.735088 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:30.735443 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:30.736119 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:30.737543 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:30.739488 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:30.740405 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:30.740541 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:30.742006 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:30.742014 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:30.742028 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:30.742641 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:30.742719 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:30.742740 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:30.742879 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:30.743113 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 06:55:30.743419 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":361,"threads_started":1}
I20260619 06:55:30.750042 4970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } }
I20260619 06:55:30.750419 4970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:30.750537 4970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 631cd097a0bd41d0aebf72278b2e6302, State: Initialized, Role: FOLLOWER
I20260619 06:55:30.751124 4970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } }
I20260619 06:55:30.751322 4970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:30.751483 4970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:30.751653 4970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:30.754230 4970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } }
I20260619 06:55:30.754743 4970 leader_election.cc:302] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 631cd097a0bd41d0aebf72278b2e6302; no voters:
I20260619 06:55:30.755331 4970 leader_election.cc:288] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:30.755501 4975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:30.756551 4975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [term 1 LEADER]: Becoming Leader. State: Replica: 631cd097a0bd41d0aebf72278b2e6302, State: Running, Role: LEADER
I20260619 06:55:30.757414 4975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } }
I20260619 06:55:30.757650 4970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:30.760413 4977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 631cd097a0bd41d0aebf72278b2e6302. Latest consensus state: current_term: 1 leader_uuid: "631cd097a0bd41d0aebf72278b2e6302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } } }
I20260619 06:55:30.761083 4977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:30.761358 4976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "631cd097a0bd41d0aebf72278b2e6302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "631cd097a0bd41d0aebf72278b2e6302" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35829 } } }
I20260619 06:55:30.762073 4976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:30.763054 4984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:30.767798 4984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:30.777197 4984 catalog_manager.cc:1347] Generated new cluster ID: 06bb137ae0d14627b5aa9de98cf636ba
I20260619 06:55:30.777365 4984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:30.809926 4984 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:30.811812 4984 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:30.818819 4984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: Generated new TSK 0
I20260619 06:55:30.819695 4984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:30.825668 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--builtin_ntp_servers=127.18.39.84:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:31.193943 4994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:31.194254 4994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:31.194340 4994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:31.201807 4994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:31.201931 4994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:31.202042 4994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:31.211298 4994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:31.212548 4994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:31.214391 4994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:31.227097 4999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:31.227133 5002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:31.227097 5000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:31.227980 4994 server_base.cc:1034] running on GCE node
I20260619 06:55:31.228636 4994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:31.229653 4994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:31.230887 4994 hybrid_clock.cc:648] HybridClock initialized: now 1781852131230846 us; error 32 us; skew 500 ppm
I20260619 06:55:31.231261 4994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:31.233572 4994 webserver.cc:460] Webserver started at http://127.18.39.65:37089/ using document root <none> and password file <none>
I20260619 06:55:31.234237 4994 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:31.234349 4994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:31.234660 4994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:31.237478 4994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "d2b9d97638a64630a3197eeae96fbf12"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "955974d1790415a3a1f217930a8215c4"
server_key_iv: "7f4e0d8d9de3e23db2326724fae7a1d6"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.238267 4994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "d2b9d97638a64630a3197eeae96fbf12"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "955974d1790415a3a1f217930a8215c4"
server_key_iv: "7f4e0d8d9de3e23db2326724fae7a1d6"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.243561 4994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:31.247123 5008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.248477 4994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:55:31.248658 4994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "d2b9d97638a64630a3197eeae96fbf12"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "955974d1790415a3a1f217930a8215c4"
server_key_iv: "7f4e0d8d9de3e23db2326724fae7a1d6"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.248883 4994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:31.287973 4994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:31.288839 4994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:31.289139 4994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:31.290179 4994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:31.291635 4994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:31.291786 4994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.291906 4994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:31.291993 4994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.325073 4994 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:40327
I20260619 06:55:31.325131 5120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:40327 every 8 connection(s)
I20260619 06:55:31.326496 4994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:31.335721 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 4994
I20260619 06:55:31.335904 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:31.336385 18589 external_mini_cluster.cc:1428] Setting key bf735efb532e3f898bd83db920a83fee
I20260619 06:55:31.339221 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:31.329022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:51773 (local address 127.18.39.126:35829)
0619 06:55:31.329520 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:31.329524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:31.330580 (+ 1056us) server_negotiation.cc:407] Connection header received
0619 06:55:31.332398 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:31.332403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:31.332499 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:31.332640 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:31.333771 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.334494 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.336191 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.336651 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.336787 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:31.338410 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:31.338416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:31.338420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:31.338442 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:31.338493 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:31.338513 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:31.338603 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:31.338873 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 06:55:31.338953 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":253,"threads_started":1}
I20260619 06:55:31.339627 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--builtin_ntp_servers=127.18.39.84:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:31.340677 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:31.329556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35829 (local address 127.18.39.65:51773)
0619 06:55:31.330418 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:31.330455 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:31.332129 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:31.332871 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:31.332880 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:31.332967 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:31.333595 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:31.333613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.334674 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.334678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:31.336054 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:31.336066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.336891 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.336898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:31.337201 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:31.338030 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:31.338032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:31.338073 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:31.338207 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:31.338626 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:31.338638 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:31.338646 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:55:31.338834 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 06:55:31.339268 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":258,"threads_started":1}
I20260619 06:55:31.342275 5121 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35829
I20260619 06:55:31.342604 5121 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:31.343384 5121 heartbeater.cc:511] Master 127.18.39.126:35829 requested a full tablet report, sending...
I20260619 06:55:31.345427 4935 ts_manager.cc:194] Registered new tserver with Master: d2b9d97638a64630a3197eeae96fbf12 (127.18.39.65:40327)
I20260619 06:55:31.346873 4935 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.65:51773
W20260619 06:55:31.671342 5125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:31.671638 5125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:31.671775 5125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:31.679172 5125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:31.679296 5125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:31.679467 5125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:31.688941 5125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:31.690274 5125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:31.692086 5125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:31.705379 5133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:31.705462 5130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:31.705430 5131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:31.706271 5125 server_base.cc:1034] running on GCE node
I20260619 06:55:31.706830 5125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:31.707726 5125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:31.708958 5125 hybrid_clock.cc:648] HybridClock initialized: now 1781852131708905 us; error 45 us; skew 500 ppm
I20260619 06:55:31.709291 5125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:31.711553 5125 webserver.cc:460] Webserver started at http://127.18.39.66:42245/ using document root <none> and password file <none>
I20260619 06:55:31.712278 5125 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:31.712409 5125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:31.712736 5125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:31.715643 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "1ccb603a645d4adb8f00f31ff489ff74"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "150ddd80075cd6567a31204f8a70e7b3"
server_key_iv: "98a363779150a265f8540d2f4493c950"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.716501 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "1ccb603a645d4adb8f00f31ff489ff74"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "150ddd80075cd6567a31204f8a70e7b3"
server_key_iv: "98a363779150a265f8540d2f4493c950"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.722777 5125 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260619 06:55:31.726368 5139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.727787 5125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:55:31.727950 5125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "1ccb603a645d4adb8f00f31ff489ff74"
format_stamp: "Formatted at 2026-06-19 06:55:31 on dist-test-slave-1h5s"
server_key: "150ddd80075cd6567a31204f8a70e7b3"
server_key_iv: "98a363779150a265f8540d2f4493c950"
server_key_version: "encryptionkey@0"
I20260619 06:55:31.728173 5125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:31.746848 5125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:31.747705 5125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:31.748019 5125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:31.749024 5125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:31.750485 5125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:31.750573 5125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.750747 5125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:31.750833 5125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:31.785096 5125 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:45171
I20260619 06:55:31.785172 5251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:45171 every 8 connection(s)
I20260619 06:55:31.786464 5125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:31.795188 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5125
I20260619 06:55:31.795405 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:31.795858 18589 external_mini_cluster.cc:1428] Setting key 3f27f7aa2d76fc7c501b0a65a05acd99
I20260619 06:55:31.798498 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:31.788897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:46015 (local address 127.18.39.126:35829)
0619 06:55:31.789074 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:31.789078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:31.790313 (+ 1235us) server_negotiation.cc:407] Connection header received
0619 06:55:31.792091 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:31.792096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:31.792202 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:31.792315 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:31.793352 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.794074 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.795724 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.796176 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.796308 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:31.797727 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:31.797731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:31.797735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:31.797751 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:31.797781 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:31.797793 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:31.797886 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:31.798098 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 06:55:31.798201 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:55:31.798530 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--builtin_ntp_servers=127.18.39.84:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:31.799582 5254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:31.789273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35829 (local address 127.18.39.66:46015)
0619 06:55:31.790149 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:31.790190 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:31.791820 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:31.792525 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:31.792533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:31.792629 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:31.793174 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:31.793189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.794229 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.794236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:31.795589 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:31.795601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:31.796391 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:31.796398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:31.796626 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:31.797392 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:31.797396 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:31.797445 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:31.797527 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:31.797885 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:31.797897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:31.797907 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:55:31.798090 (+ 183us) client_negotiation.cc:240] Negotiation successful
0619 06:55:31.798489 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":247,"threads_started":1}
I20260619 06:55:31.801090 5252 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35829
I20260619 06:55:31.801447 5252 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:31.802357 5252 heartbeater.cc:511] Master 127.18.39.126:35829 requested a full tablet report, sending...
I20260619 06:55:31.804087 4935 ts_manager.cc:194] Registered new tserver with Master: 1ccb603a645d4adb8f00f31ff489ff74 (127.18.39.66:45171)
I20260619 06:55:31.805327 4935 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.66:46015
W20260619 06:55:32.131940 5255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:32.132231 5255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:32.132421 5255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:32.139884 5255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:32.140007 5255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:32.140187 5255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:32.149721 5255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:35829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:32.151100 5255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:32.152969 5255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:32.166734 5260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:32.166690 5261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:32.166797 5263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:32.167842 5255 server_base.cc:1034] running on GCE node
I20260619 06:55:32.168437 5255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:32.169462 5255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:32.170699 5255 hybrid_clock.cc:648] HybridClock initialized: now 1781852132170637 us; error 67 us; skew 500 ppm
I20260619 06:55:32.171060 5255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:32.173492 5255 webserver.cc:460] Webserver started at http://127.18.39.67:33621/ using document root <none> and password file <none>
I20260619 06:55:32.174225 5255 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:32.174357 5255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:32.174696 5255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:32.177860 5255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "dfd65bc168a4478d9906dbc4c4953f4f"
format_stamp: "Formatted at 2026-06-19 06:55:32 on dist-test-slave-1h5s"
server_key: "ed0a9b93cb27002a8b673bdc4a66add5"
server_key_iv: "8a8509368474431809608223f04dc266"
server_key_version: "encryptionkey@0"
I20260619 06:55:32.178764 5255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "dfd65bc168a4478d9906dbc4c4953f4f"
format_stamp: "Formatted at 2026-06-19 06:55:32 on dist-test-slave-1h5s"
server_key: "ed0a9b93cb27002a8b673bdc4a66add5"
server_key_iv: "8a8509368474431809608223f04dc266"
server_key_version: "encryptionkey@0"
I20260619 06:55:32.184443 5255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:55:32.188320 5269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:32.189913 5255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:55:32.190114 5255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "dfd65bc168a4478d9906dbc4c4953f4f"
format_stamp: "Formatted at 2026-06-19 06:55:32 on dist-test-slave-1h5s"
server_key: "ed0a9b93cb27002a8b673bdc4a66add5"
server_key_iv: "8a8509368474431809608223f04dc266"
server_key_version: "encryptionkey@0"
I20260619 06:55:32.190378 5255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:32.213663 5255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:32.214529 5255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:32.214866 5255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:32.215906 5255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:32.217481 5255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:32.217607 5255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:32.217744 5255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:32.217840 5255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:32.251196 5255 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:45533
I20260619 06:55:32.251286 5381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:45533 every 8 connection(s)
I20260619 06:55:32.252597 5255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:32.254441 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5255
I20260619 06:55:32.254622 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:32.255125 18589 external_mini_cluster.cc:1428] Setting key c720b1b9e10d2a00a14d11f6604c87ff
I20260619 06:55:32.265924 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.255516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60625 (local address 127.18.39.126:35829)
0619 06:55:32.255737 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.255743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.257629 (+ 1886us) server_negotiation.cc:407] Connection header received
0619 06:55:32.259461 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.259467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.259556 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.259676 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:32.260770 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.261669 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.263199 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.263600 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.263729 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.265000 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:32.265006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:32.265011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:32.265037 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:32.265069 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:32.265080 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:32.265170 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.265484 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.265582 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:55:32.266907 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.256177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:35829 (local address 127.18.39.67:60625)
0619 06:55:32.257467 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.257519 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.259202 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.259901 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.259910 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.259993 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:32.260601 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.260617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.261839 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.261844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.263051 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.263062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.263828 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.263837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.264067 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.264688 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:32.264691 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:32.264730 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:32.264812 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:32.265183 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:32.265195 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:32.265211 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.265473 (+ 262us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.265862 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":373,"threads_started":1}
I20260619 06:55:32.268236 5382 heartbeater.cc:347] Connected to a master server at 127.18.39.126:35829
I20260619 06:55:32.268576 5382 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:32.269441 5382 heartbeater.cc:511] Master 127.18.39.126:35829 requested a full tablet report, sending...
I20260619 06:55:32.270843 4935 ts_manager.cc:194] Registered new tserver with Master: dfd65bc168a4478d9906dbc4c4953f4f (127.18.39.67:45533)
I20260619 06:55:32.271634 4935 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.67:60625
I20260619 06:55:32.282673 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:32.292369 5123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.286816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.18.39.126:35829)
0619 06:55:32.287025 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.287030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.287437 (+ 407us) server_negotiation.cc:407] Connection header received
0619 06:55:32.287573 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.287577 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.287654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.287740 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:32.288206 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.288905 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.290269 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.290666 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.290760 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.291739 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:32.291743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:32.291746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:32.291763 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:32.291795 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:32.291806 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:32.291908 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.292066 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.292110 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:55:32.296299 4935 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:32.303812 4935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:32.329423 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.316490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.18.39.66:45171)
0619 06:55:32.317989 (+ 1499us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.317995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.318019 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:55:32.318093 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.318098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.318279 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.318467 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:32.319220 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.322594 (+ 3374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.326252 (+ 3658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.327692 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.327818 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.328073 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.328841 (+ 768us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.329022 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1093,"thread_start_us":975,"threads_started":1}
I20260619 06:55:32.329890 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.316074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:45533 (local address 127.0.0.1:40948)
0619 06:55:32.319440 (+ 3366us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.319460 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.319567 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.320661 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.320669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.320704 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:32.321225 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.321239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.323654 (+ 2415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.323660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.326027 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.326043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.328198 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.328204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.328502 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.329177 (+ 675us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.329254 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.329507 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3225,"mutex_wait_us":104,"thread_start_us":3082,"threads_started":1}
I20260619 06:55:32.330577 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.316062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:45171 (local address 127.0.0.1:34272)
0619 06:55:32.316906 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.316940 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.317107 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.318481 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.318488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.318518 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:32.318988 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.319003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.323136 (+ 4133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.323142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.326007 (+ 2865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.326026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.328248 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.328255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.328537 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.328602 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.330081 (+ 1479us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.330228 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":69,"thread_start_us":491,"threads_started":1}
I20260619 06:55:32.330919 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.316171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.18.39.67:45533)
0619 06:55:32.319133 (+ 2962us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.319143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.319688 (+ 545us) server_negotiation.cc:407] Connection header received
0619 06:55:32.319811 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.319821 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.320092 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.320318 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:32.321530 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.323518 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.326274 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.327678 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.327811 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.328069 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.330369 (+ 2300us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.330548 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2786,"thread_start_us":2627,"threads_started":1}
I20260619 06:55:32.334029 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.319556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:40327 (local address 127.0.0.1:49298)
0619 06:55:32.323593 (+ 4037us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.323611 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.323825 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.325082 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.325087 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.325110 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:32.325583 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.325594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.327695 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.327699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.330616 (+ 2917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.330634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.332918 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.332924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.333091 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.333120 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.333611 (+ 491us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.333751 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3903,"thread_start_us":3751,"threads_started":1}
I20260619 06:55:32.334228 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.319837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.18.39.65:40327)
0619 06:55:32.323126 (+ 3289us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.323134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.323706 (+ 572us) server_negotiation.cc:407] Connection header received
0619 06:55:32.324027 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.324035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.324304 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.324473 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:32.325792 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.327488 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.330827 (+ 3339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.332310 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.332428 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.332733 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.333693 (+ 960us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.333869 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3151,"thread_start_us":3044,"threads_started":1}
I20260619 06:55:32.334731 5317 tablet_service.cc:1467] Processing CreateTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 (DEFAULT_TABLE table=test-table [id=10d4f2d403214c08be1d79cce580fbf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:32.334717 5187 tablet_service.cc:1467] Processing CreateTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 (DEFAULT_TABLE table=test-table [id=10d4f2d403214c08be1d79cce580fbf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:32.336143 5187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d2dc3f11974aa682098e20a08ed8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:32.336175 5317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d2dc3f11974aa682098e20a08ed8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:32.338608 5056 tablet_service.cc:1467] Processing CreateTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 (DEFAULT_TABLE table=test-table [id=10d4f2d403214c08be1d79cce580fbf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:32.340041 5056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8d2dc3f11974aa682098e20a08ed8e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:32.347509 5400 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Bootstrap starting.
I20260619 06:55:32.349947 5121 heartbeater.cc:503] Master 127.18.39.126:35829 was elected leader, sending a full tablet report...
I20260619 06:55:32.352089 5400 tablet_bootstrap.cc:654] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:32.353734 5400 log.cc:824] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:32.356880 5401 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Bootstrap starting.
I20260619 06:55:32.364866 5401 tablet_bootstrap.cc:654] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:32.366020 5401 log.cc:824] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:32.366942 5403 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Bootstrap starting.
I20260619 06:55:32.367941 5400 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: No bootstrap required, opened a new log
I20260619 06:55:32.368227 5400 ts_tablet_manager.cc:1397] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.005s
I20260619 06:55:32.370370 5403 tablet_bootstrap.cc:654] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:32.371690 5403 log.cc:824] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:32.373585 5401 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: No bootstrap required, opened a new log
I20260619 06:55:32.373905 5401 ts_tablet_manager.cc:1397] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.000s
I20260619 06:55:32.374480 5403 tablet_bootstrap.cc:492] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: No bootstrap required, opened a new log
I20260619 06:55:32.374740 5403 ts_tablet_manager.cc:1397] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:55:32.378571 5400 raft_consensus.cc:348] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.379068 5400 raft_consensus.cc:374] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:32.379199 5400 raft_consensus.cc:729] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ccb603a645d4adb8f00f31ff489ff74, State: Initialized, Role: FOLLOWER
I20260619 06:55:32.379901 5400 consensus_queue.cc:260] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.381131 5252 heartbeater.cc:503] Master 127.18.39.126:35829 was elected leader, sending a full tablet report...
I20260619 06:55:32.381892 5400 ts_tablet_manager.cc:1428] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260619 06:55:32.383890 5403 raft_consensus.cc:348] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.384495 5403 raft_consensus.cc:374] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:32.384668 5403 raft_consensus.cc:729] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b9d97638a64630a3197eeae96fbf12, State: Initialized, Role: FOLLOWER
I20260619 06:55:32.384429 5401 raft_consensus.cc:348] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.384847 5401 raft_consensus.cc:374] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:32.384968 5401 raft_consensus.cc:729] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfd65bc168a4478d9906dbc4c4953f4f, State: Initialized, Role: FOLLOWER
I20260619 06:55:32.385480 5403 consensus_queue.cc:260] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.385564 5401 consensus_queue.cc:260] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.387028 5382 heartbeater.cc:503] Master 127.18.39.126:35829 was elected leader, sending a full tablet report...
I20260619 06:55:32.387684 5401 ts_tablet_manager.cc:1428] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 06:55:32.389685 5403 ts_tablet_manager.cc:1428] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260619 06:55:32.422315 5407 raft_consensus.cc:482] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:32.422735 5407 raft_consensus.cc:504] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.424499 5407 leader_election.cc:288] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfd65bc168a4478d9906dbc4c4953f4f (127.18.39.67:45533), 1ccb603a645d4adb8f00f31ff489ff74 (127.18.39.66:45171)
I20260619 06:55:32.431689 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.424827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:33805 (local address 127.18.39.67:45533)
0619 06:55:32.425479 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.425488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.425642 (+ 154us) server_negotiation.cc:407] Connection header received
0619 06:55:32.425890 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.425896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.425982 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.426092 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:32.426972 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.427768 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.429814 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.430628 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.430758 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.430809 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.431219 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.431398 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517}
I20260619 06:55:32.431725 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.424660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:45533 (local address 127.18.39.65:33805)
0619 06:55:32.425492 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.425522 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.425712 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.426235 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.426240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.426270 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:32.426688 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.426701 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.427918 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.427924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.429616 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.429632 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.430814 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.430823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.431007 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.431048 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.431130 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.431262 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":38,"thread_start_us":385,"threads_started":1}
I20260619 06:55:32.431797 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.425232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:43141 (local address 127.18.39.66:45171)
0619 06:55:32.425458 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.425464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.426115 (+ 651us) server_negotiation.cc:407] Connection header received
0619 06:55:32.426298 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.426306 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.426407 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.426525 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:32.427179 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.427913 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.429927 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.430781 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.430952 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.431021 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.431370 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.431498 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:55:32.431951 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.425089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:45171 (local address 127.18.39.65:43141)
0619 06:55:32.425990 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:32.426006 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:32.426147 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:32.426541 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:32.426544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:32.426566 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:32.426998 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.427009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.428088 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.428092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.429714 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:32.429751 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.431005 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.431010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:32.431183 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.431241 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 06:55:32.431326 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 06:55:32.431445 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":4,"thread_start_us":524,"threads_started":1}
I20260619 06:55:32.433000 5207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d2dc3f11974aa682098e20a08ed8e1" candidate_uuid: "d2b9d97638a64630a3197eeae96fbf12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ccb603a645d4adb8f00f31ff489ff74" is_pre_election: true
I20260619 06:55:32.433000 5337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d2dc3f11974aa682098e20a08ed8e1" candidate_uuid: "d2b9d97638a64630a3197eeae96fbf12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" is_pre_election: true
I20260619 06:55:32.433552 5337 raft_consensus.cc:2454] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2b9d97638a64630a3197eeae96fbf12 in term 0.
I20260619 06:55:32.433552 5207 raft_consensus.cc:2454] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2b9d97638a64630a3197eeae96fbf12 in term 0.
I20260619 06:55:32.434240 5012 leader_election.cc:302] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1ccb603a645d4adb8f00f31ff489ff74, d2b9d97638a64630a3197eeae96fbf12, dfd65bc168a4478d9906dbc4c4953f4f; no voters:
I20260619 06:55:32.434665 5407 raft_consensus.cc:2781] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:32.434820 5407 raft_consensus.cc:482] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:32.434937 5407 raft_consensus.cc:3037] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:32.437218 5407 raft_consensus.cc:504] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.437978 5407 leader_election.cc:288] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [CANDIDATE]: Term 1 election: Requested vote from peers dfd65bc168a4478d9906dbc4c4953f4f (127.18.39.67:45533), 1ccb603a645d4adb8f00f31ff489ff74 (127.18.39.66:45171)
I20260619 06:55:32.438512 5337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d2dc3f11974aa682098e20a08ed8e1" candidate_uuid: "d2b9d97638a64630a3197eeae96fbf12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfd65bc168a4478d9906dbc4c4953f4f"
I20260619 06:55:32.438587 5207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8d2dc3f11974aa682098e20a08ed8e1" candidate_uuid: "d2b9d97638a64630a3197eeae96fbf12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ccb603a645d4adb8f00f31ff489ff74"
I20260619 06:55:32.438768 5337 raft_consensus.cc:3037] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:32.438854 5207 raft_consensus.cc:3037] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:32.441123 5337 raft_consensus.cc:2454] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2b9d97638a64630a3197eeae96fbf12 in term 1.
I20260619 06:55:32.441211 5207 raft_consensus.cc:2454] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2b9d97638a64630a3197eeae96fbf12 in term 1.
I20260619 06:55:32.441659 5012 leader_election.cc:302] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2b9d97638a64630a3197eeae96fbf12, dfd65bc168a4478d9906dbc4c4953f4f; no voters:
I20260619 06:55:32.442003 5407 raft_consensus.cc:2781] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:32.442502 5407 raft_consensus.cc:686] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 1 LEADER]: Becoming Leader. State: Replica: d2b9d97638a64630a3197eeae96fbf12, State: Running, Role: LEADER
I20260619 06:55:32.443010 5407 consensus_queue.cc:237] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } }
I20260619 06:55:32.448022 4934 catalog_manager.cc:5495] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2b9d97638a64630a3197eeae96fbf12 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "d2b9d97638a64630a3197eeae96fbf12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2b9d97638a64630a3197eeae96fbf12" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 40327 } health_report { overall_health: HEALTHY } } }
I20260619 06:55:32.489352 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.484110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.18.39.65:40327)
0619 06:55:32.484325 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.484330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.484347 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:55:32.484452 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.484456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.484545 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.484712 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:32.485079 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.485870 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.487235 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.487642 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.487742 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.488254 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:32.488794 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:32.488929 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.489078 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.489117 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:32.498231 5337 raft_consensus.cc:1261] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 1 FOLLOWER]: Refusing update from remote peer d2b9d97638a64630a3197eeae96fbf12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:32.498230 5207 raft_consensus.cc:1261] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 1 FOLLOWER]: Refusing update from remote peer d2b9d97638a64630a3197eeae96fbf12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:32.499260 5411 consensus_queue.cc:1035] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfd65bc168a4478d9906dbc4c4953f4f" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:32.499619 5407 consensus_queue.cc:1035] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ccb603a645d4adb8f00f31ff489ff74" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:55:32.504482 5383 tablet.cc:2369] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:32.511736 5414 mvcc.cc:205] Tried to move back new op lower bound from 7298466334701969408 to 7298466334490939392. Current Snapshot: MvccSnapshot[applied={T|T < 7298466334701969408}]
I20260619 06:55:32.530910 4934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:32.531213 4934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:32.535285 4934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 631cd097a0bd41d0aebf72278b2e6302: Sending DeleteTablet for 3 replicas of tablet a8d2dc3f11974aa682098e20a08ed8e1
I20260619 06:55:32.536720 5187 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:32 UTC) from {username='slave'} at 127.0.0.1:34272
I20260619 06:55:32.537142 5056 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:32 UTC) from {username='slave'} at 127.0.0.1:49298
I20260619 06:55:32.537596 5317 tablet_service.cc:1514] Processing DeleteTablet for tablet a8d2dc3f11974aa682098e20a08ed8e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:32 UTC) from {username='slave'} at 127.0.0.1:40948
I20260619 06:55:32.538559 5425 tablet_replica.cc:330] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: stopping tablet replica
I20260619 06:55:32.539484 5425 raft_consensus.cc:2229] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:32.540127 5425 raft_consensus.cc:2258] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:32.540480 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4994
I20260619 06:55:32.540495 5426 tablet_replica.cc:330] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: stopping tablet replica
I20260619 06:55:32.540894 5426 raft_consensus.cc:2229] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:32.541376 5426 raft_consensus.cc:2258] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:32.542146 5427 tablet_replica.cc:330] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: stopping tablet replica
I20260619 06:55:32.542531 5427 raft_consensus.cc:2229] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:32.542811 5425 ts_tablet_manager.cc:1905] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:32.543253 5427 raft_consensus.cc:2258] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:32.545733 5426 ts_tablet_manager.cc:1905] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:32.546185 5427 ts_tablet_manager.cc:1905] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:32.549801 5425 ts_tablet_manager.cc:1918] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:32.550060 5425 log.cc:1196] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/a8d2dc3f11974aa682098e20a08ed8e1
I20260619 06:55:32.550685 5425 ts_tablet_manager.cc:1939] T a8d2dc3f11974aa682098e20a08ed8e1 P 1ccb603a645d4adb8f00f31ff489ff74: Deleting consensus metadata
I20260619 06:55:32.552758 4920 catalog_manager.cc:4841] TS 1ccb603a645d4adb8f00f31ff489ff74 (127.18.39.66:45171): tablet a8d2dc3f11974aa682098e20a08ed8e1 (table test-table [id=10d4f2d403214c08be1d79cce580fbf5]) successfully deleted
I20260619 06:55:32.553092 5426 ts_tablet_manager.cc:1918] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:32.553431 5426 log.cc:1196] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/a8d2dc3f11974aa682098e20a08ed8e1
I20260619 06:55:32.553918 5426 ts_tablet_manager.cc:1939] T a8d2dc3f11974aa682098e20a08ed8e1 P dfd65bc168a4478d9906dbc4c4953f4f: Deleting consensus metadata
I20260619 06:55:32.555804 4922 catalog_manager.cc:4841] TS dfd65bc168a4478d9906dbc4c4953f4f (127.18.39.67:45533): tablet a8d2dc3f11974aa682098e20a08ed8e1 (table test-table [id=10d4f2d403214c08be1d79cce580fbf5]) successfully deleted
I20260619 06:55:32.563170 5427 ts_tablet_manager.cc:1918] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:32.563447 5427 log.cc:1196] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/a8d2dc3f11974aa682098e20a08ed8e1
I20260619 06:55:32.564066 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.544053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49322 (local address 127.18.39.65:40327)
0619 06:55:32.544240 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.544246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.552781 (+ 8535us) server_negotiation.cc:407] Connection header received
0619 06:55:32.553317 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.553324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.553451 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.553530 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:32.554242 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.555505 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.557033 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.557626 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.557772 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.562322 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:32.562336 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:32.562359 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:32.563114 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:32.563186 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:32.563208 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:32.563348 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.563550 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.563672 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 06:55:32.564116 5427 ts_tablet_manager.cc:1939] T a8d2dc3f11974aa682098e20a08ed8e1 P d2b9d97638a64630a3197eeae96fbf12: Deleting consensus metadata
I20260619 06:55:32.564827 5116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:32.664633 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4994
I20260619 06:55:32.664830 4922 catalog_manager.cc:4841] TS d2b9d97638a64630a3197eeae96fbf12 (127.18.39.65:40327): tablet a8d2dc3f11974aa682098e20a08ed8e1 (table test-table [id=10d4f2d403214c08be1d79cce580fbf5]) successfully deleted
I20260619 06:55:32.683344 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5125
I20260619 06:55:32.690694 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.684213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34278 (local address 127.18.39.66:45171)
0619 06:55:32.684440 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.684445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.684462 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:55:32.684519 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.684523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.684610 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.684738 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:32.685133 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.685979 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.687323 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.687847 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.687986 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.689124 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:32.689134 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:32.689156 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:32.689895 (+ 739us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:32.689953 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:32.689973 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:32.690101 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.690236 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.690351 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260619 06:55:32.691382 5247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:32.783361 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5125
I20260619 06:55:32.801986 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5255
I20260619 06:55:32.808751 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:32.802807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.18.39.67:45533)
0619 06:55:32.802978 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:32.802982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:32.803004 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:55:32.803173 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:32.803177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:32.803262 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:32.803351 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:32.803742 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.804484 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.805808 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:32.806206 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:32.806301 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:32.807385 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:32.807393 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:32.807408 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:32.808007 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:32.808055 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:32.808073 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:32.808159 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:32.808327 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 06:55:32.808436 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:55:32.809484 5377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:32.899744 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5255
I20260619 06:55:32.916429 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 4904
I20260619 06:55:32.917675 4965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:33.029424 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 4904
2026-06-19T06:55:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2822 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:33 dist-test-slave-1h5s krb5kdc[5435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:33 dist-test-slave-1h5s krb5kdc[5435](info): set up 2 sockets
Jun 19 06:55:33 dist-test-slave-1h5s krb5kdc[5435](info): commencing operation
krb5kdc: starting...
W20260619 06:55:34.149739 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:34 dist-test-slave-1h5s krb5kdc[5435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:55:34.406610 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:44361
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:44361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:34.742372 5448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:34.742671 5448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:34.742765 5448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:34.750065 5448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:34.750183 5448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:34.750254 5448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:34.750314 5448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:34.750360 5448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:34.760433 5448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:44361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:44361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:34.761860 5448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:34.763289 5448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:34.775758 5456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:34.775689 5454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:34.775794 5453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:34.776283 5448 server_base.cc:1034] running on GCE node
I20260619 06:55:34.777190 5448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:34.778707 5448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:34.779912 5448 hybrid_clock.cc:648] HybridClock initialized: now 1781852134779875 us; error 35 us; skew 500 ppm
I20260619 06:55:34.780275 5448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:34 dist-test-slave-1h5s krb5kdc[5435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:34.783501 5448 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:55:34.785072 5448 webserver.cc:460] Webserver started at http://127.18.39.126:38085/ using document root <none> and password file <none>
I20260619 06:55:34.785841 5448 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:34.786005 5448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:34.786317 5448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:34.789417 5448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "e81077f6594d4783966f5f37577cb7f0"
format_stamp: "Formatted at 2026-06-19 06:55:34 on dist-test-slave-1h5s"
server_key: "fa7a8d2828b9942dfeb62bec0c185157"
server_key_iv: "e62f7e367f2e0c1277b688f67770765b"
server_key_version: "encryptionkey@0"
I20260619 06:55:34.790222 5448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "e81077f6594d4783966f5f37577cb7f0"
format_stamp: "Formatted at 2026-06-19 06:55:34 on dist-test-slave-1h5s"
server_key: "fa7a8d2828b9942dfeb62bec0c185157"
server_key_iv: "e62f7e367f2e0c1277b688f67770765b"
server_key_version: "encryptionkey@0"
I20260619 06:55:34.795719 5448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:34.799450 5463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:34.800977 5448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:34.801180 5448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "e81077f6594d4783966f5f37577cb7f0"
format_stamp: "Formatted at 2026-06-19 06:55:34 on dist-test-slave-1h5s"
server_key: "fa7a8d2828b9942dfeb62bec0c185157"
server_key_iv: "e62f7e367f2e0c1277b688f67770765b"
server_key_version: "encryptionkey@0"
I20260619 06:55:34.801514 5448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:34.829926 5448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:34.834285 5448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:34.834599 5448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:34.852701 5448 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:44361
I20260619 06:55:34.852712 5514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:44361 every 8 connection(s)
I20260619 06:55:34.854220 5448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:34.858359 5515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:34.861680 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5448
I20260619 06:55:34.861874 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:34.862371 18589 external_mini_cluster.cc:1428] Setting key d050a7020293be07d49c01c626327b7d
I20260619 06:55:34.871295 5515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: Bootstrap starting.
I20260619 06:55:34.875232 5515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:34.876585 5515 log.cc:824] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:34.879799 5515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: No bootstrap required, opened a new log
Jun 19 06:55:34 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:34.889549 5515 raft_consensus.cc:348] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } }
I20260619 06:55:34.889963 5515 raft_consensus.cc:374] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:34.890065 5515 raft_consensus.cc:729] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e81077f6594d4783966f5f37577cb7f0, State: Initialized, Role: FOLLOWER
I20260619 06:55:34.890528 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:34.864849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.18.39.126:44361)
0619 06:55:34.865934 (+ 1085us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:34.865951 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:34.866396 (+ 445us) server_negotiation.cc:407] Connection header received
0619 06:55:34.869405 (+ 3009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:34.869441 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:34.870025 (+ 584us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:34.870491 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:34.872089 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:34.874189 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:34.875690 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:34.876766 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:34.876958 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:34.881354 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:34.881366 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:34.881386 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:34.881440 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:34.885763 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:34.886621 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:34.886629 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:34.886643 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:34.886842 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:34.887288 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:34.887293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:34.887302 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:34.887937 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:34.888296 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:34.888745 (+ 449us) server_negotiation.cc:299] Negotiation successful
0619 06:55:34.889218 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":510,"threads_started":1}
I20260619 06:55:34.890694 5515 consensus_queue.cc:260] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } }
I20260619 06:55:34.890986 5515 raft_consensus.cc:388] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:34.891144 5515 raft_consensus.cc:482] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:34.891347 5515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:34.894493 5515 raft_consensus.cc:504] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } }
I20260619 06:55:34.895004 5515 leader_election.cc:302] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e81077f6594d4783966f5f37577cb7f0; no voters:
I20260619 06:55:34.895741 5515 leader_election.cc:288] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:34.895859 5520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:34.896548 5520 raft_consensus.cc:686] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [term 1 LEADER]: Becoming Leader. State: Replica: e81077f6594d4783966f5f37577cb7f0, State: Running, Role: LEADER
I20260619 06:55:34.897123 5520 consensus_queue.cc:237] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } }
I20260619 06:55:34.898088 5515 sys_catalog.cc:564] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:34.900007 5521 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e81077f6594d4783966f5f37577cb7f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } } }
I20260619 06:55:34.900396 5521 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:34.900303 5522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e81077f6594d4783966f5f37577cb7f0. Latest consensus state: current_term: 1 leader_uuid: "e81077f6594d4783966f5f37577cb7f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81077f6594d4783966f5f37577cb7f0" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 44361 } } }
I20260619 06:55:34.900701 5522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:34.904196 5525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:34.913275 5525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:34.922642 5525 catalog_manager.cc:1347] Generated new cluster ID: a3cecdb817dc4ddb95e54118bcbeba3d
I20260619 06:55:34.922778 5525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:34.931764 5525 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:34.933250 5525 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:34.939880 5525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: Generated new TSK 0
I20260619 06:55:34.940769 5525 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.39.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
Entry for principal kudu/127.18.39.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab.
I20260619 06:55:35.012209 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:35.358104 5541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:35.358409 5541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:35.358577 5541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:35.366170 5541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:35.366290 5541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:35.366474 5541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:35.376585 5541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.65.keytab
--principal=kudu/127.18.39.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:35.377983 5541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:35.379403 5541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:35.393573 5546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:35.393625 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:35.393591 5549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:35.394832 5541 server_base.cc:1034] running on GCE node
I20260619 06:55:35.395543 5541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:35.396601 5541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:35.397867 5541 hybrid_clock.cc:648] HybridClock initialized: now 1781852135397809 us; error 50 us; skew 500 ppm
I20260619 06:55:35.398241 5541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:35 dist-test-slave-1h5s krb5kdc[5435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852135, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:35.401748 5541 init.cc:377] Logged in from keytab as kudu/127.18.39.65@KRBTEST.COM (short username kudu)
I20260619 06:55:35.403445 5541 webserver.cc:460] Webserver started at http://127.18.39.65:42487/ using document root <none> and password file <none>
I20260619 06:55:35.404145 5541 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:35.404253 5541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:35.404567 5541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:35.407845 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "2a48f9cf7cb3440bbd7dff473649cd0e"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "5e92cdd8da7a3d04ef5516976de53e13"
server_key_iv: "be8a8c6a2ec4bc0c41a35d8fe8da3482"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.408629 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "2a48f9cf7cb3440bbd7dff473649cd0e"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "5e92cdd8da7a3d04ef5516976de53e13"
server_key_iv: "be8a8c6a2ec4bc0c41a35d8fe8da3482"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.414100 5541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:55:35.417795 5556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:35.419226 5541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:55:35.419422 5541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "2a48f9cf7cb3440bbd7dff473649cd0e"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "5e92cdd8da7a3d04ef5516976de53e13"
server_key_iv: "be8a8c6a2ec4bc0c41a35d8fe8da3482"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.419641 5541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:35.443147 5541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:35.448029 5541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:35.448341 5541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:35.449321 5541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:35.450747 5541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:35.450855 5541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:35.451000 5541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:35.451072 5541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:35.483412 5541 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:38327
I20260619 06:55:35.483462 5668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:38327 every 8 connection(s)
I20260619 06:55:35.484778 5541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:35.492781 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5541
I20260619 06:55:35.492973 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:35.493496 18589 external_mini_cluster.cc:1428] Setting key 74b8e7f2f050172ec57f3cbd47cf1439
Jun 19 06:55:35 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852135, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.65@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:35.506858 5671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:35.487412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:48221 (local address 127.18.39.126:44361)
0619 06:55:35.487921 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:35.487928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:35.488863 (+ 935us) server_negotiation.cc:407] Connection header received
0619 06:55:35.490767 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:35.490772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:35.490843 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:35.490939 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:35.492446 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:35.493316 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:35.495521 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:35.496027 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:35.496135 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:35.500550 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:35.500555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:35.500558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:35.500593 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:35.503388 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:35.504190 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:35.504197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:35.504201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:35.504332 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:35.504856 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:35.504861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:35.504865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:35.505205 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:35.505438 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:35.506224 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 06:55:35.506487 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":241,"threads_started":1}
I20260619 06:55:35.507907 5672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:35.487950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:44361 (local address 127.18.39.65:48221)
0619 06:55:35.488695 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:35.488732 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:35.490505 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:35.491141 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:35.491150 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:35.491683 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:35.492257 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:35.492278 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:35.493534 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:35.493541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:35.495246 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:35.495261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:35.496292 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:35.496300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:35.496627 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:35.497424 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:35.497460 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:35.500301 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:35.503556 (+ 3255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:35.503566 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:35.503605 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:35.504055 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:35.504453 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:35.504458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:35.504462 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:35.504741 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:35.505374 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:35.505383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:35.505828 (+ 445us) client_negotiation.cc:769] Sending connection context
0619 06:55:35.506114 (+ 286us) client_negotiation.cc:240] Negotiation successful
0619 06:55:35.506712 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":293,"threads_started":1}
I20260619 06:55:35.509589 5669 heartbeater.cc:347] Connected to a master server at 127.18.39.126:44361
I20260619 06:55:35.509907 5669 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:35.510622 5669 heartbeater.cc:511] Master 127.18.39.126:44361 requested a full tablet report, sending...
I20260619 06:55:35.512948 5480 ts_manager.cc:194] Registered new tserver with Master: 2a48f9cf7cb3440bbd7dff473649cd0e (127.18.39.65:38327)
I20260619 06:55:35.515178 5480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.65@KRBTEST.COM'} at 127.18.39.65:48221
WARNING: no policy specified for kudu/127.18.39.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
Entry for principal kudu/127.18.39.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab.
I20260619 06:55:35.549803 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:35.890750 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:35.891042 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:35.891207 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:35.898734 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:35.898912 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:35.899119 5675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:35.909597 5675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.66.keytab
--principal=kudu/127.18.39.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:35.910938 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:35.912395 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:35.926198 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:35.926113 5683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:35.926148 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:35.927500 5675 server_base.cc:1034] running on GCE node
I20260619 06:55:35.928141 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:35.929121 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:35.930349 5675 hybrid_clock.cc:648] HybridClock initialized: now 1781852135930276 us; error 66 us; skew 500 ppm
I20260619 06:55:35.930665 5675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:35 dist-test-slave-1h5s krb5kdc[5435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852135, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:35.934059 5675 init.cc:377] Logged in from keytab as kudu/127.18.39.66@KRBTEST.COM (short username kudu)
I20260619 06:55:35.935616 5675 webserver.cc:460] Webserver started at http://127.18.39.66:37409/ using document root <none> and password file <none>
I20260619 06:55:35.936336 5675 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:35.936473 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:35.936832 5675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:35.939813 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "73b25ff5f95e4af483c4fb02388e5ad0"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "2a729d88ae8ec56db914acf101ec2952"
server_key_iv: "93566d8af6de3d37a287e7a010fe4df9"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.940588 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "73b25ff5f95e4af483c4fb02388e5ad0"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "2a729d88ae8ec56db914acf101ec2952"
server_key_iv: "93566d8af6de3d37a287e7a010fe4df9"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.945825 5675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:55:35.949257 5690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:35.950641 5675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:35.950831 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "73b25ff5f95e4af483c4fb02388e5ad0"
format_stamp: "Formatted at 2026-06-19 06:55:35 on dist-test-slave-1h5s"
server_key: "2a729d88ae8ec56db914acf101ec2952"
server_key_iv: "93566d8af6de3d37a287e7a010fe4df9"
server_key_version: "encryptionkey@0"
I20260619 06:55:35.951045 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:35.962817 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:35.967265 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:35.967576 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:35.968568 5675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:35.970203 5675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:35.970309 5675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:35.970458 5675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:35.970534 5675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:36.004954 5675 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:45675
I20260619 06:55:36.005033 5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:45675 every 8 connection(s)
I20260619 06:55:36.006543 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:36.014189 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5675
I20260619 06:55:36.014381 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:36.014876 18589 external_mini_cluster.cc:1428] Setting key 0058b7a284a4ef47933e86db2bc60378
Jun 19 06:55:36 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852135, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.66@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:36.027854 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.009213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:39749 (local address 127.18.39.126:44361)
0619 06:55:36.009713 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.009719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.010735 (+ 1016us) server_negotiation.cc:407] Connection header received
0619 06:55:36.012607 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.012614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.012713 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.012836 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:36.014461 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.015241 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.017021 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.017527 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.017660 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.021474 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:36.021479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:36.021483 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:36.021515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:36.024466 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.025312 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.025319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.025323 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.025511 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.025938 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.025945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.025950 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.026309 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:36.026475 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.027397 (+ 922us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.027571 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":249,"threads_started":1}
I20260619 06:55:36.029515 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.009848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:44361 (local address 127.18.39.66:39749)
0619 06:55:36.010563 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.010614 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.012318 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.013065 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.013073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.013669 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:36.014221 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.014238 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.015392 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.015404 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.016886 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.016898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.017750 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.017757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.017982 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.018730 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:36.018752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:36.021210 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:36.024614 (+ 3404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:36.024623 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:36.024642 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:36.025188 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:36.025623 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:36.025629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:36.025633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:36.025824 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:36.026506 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:36.026518 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:36.027017 (+ 499us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.027333 (+ 316us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.027964 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":275,"threads_started":1}
I20260619 06:55:36.031085 5803 heartbeater.cc:347] Connected to a master server at 127.18.39.126:44361
I20260619 06:55:36.031409 5803 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:36.032107 5803 heartbeater.cc:511] Master 127.18.39.126:44361 requested a full tablet report, sending...
I20260619 06:55:36.033802 5480 ts_manager.cc:194] Registered new tserver with Master: 73b25ff5f95e4af483c4fb02388e5ad0 (127.18.39.66:45675)
I20260619 06:55:36.035068 5480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.66@KRBTEST.COM'} at 127.18.39.66:39749
WARNING: no policy specified for kudu/127.18.39.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
Entry for principal kudu/127.18.39.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab.
I20260619 06:55:36.085155 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:36.424613 5809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:36.424917 5809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:36.425014 5809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:36.432631 5809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:36.432781 5809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:36.432896 5809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:36.443104 5809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.67.keytab
--principal=kudu/127.18.39.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:44361
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:36.444461 5809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:36.445914 5809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:36.460052 5817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:36.460271 5814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:36.460078 5815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:36.461305 5809 server_base.cc:1034] running on GCE node
I20260619 06:55:36.462033 5809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:36.462957 5809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:36.464180 5809 hybrid_clock.cc:648] HybridClock initialized: now 1781852136464133 us; error 37 us; skew 500 ppm
I20260619 06:55:36.464466 5809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:36 dist-test-slave-1h5s krb5kdc[5435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852136, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:36.467855 5809 init.cc:377] Logged in from keytab as kudu/127.18.39.67@KRBTEST.COM (short username kudu)
I20260619 06:55:36.469488 5809 webserver.cc:460] Webserver started at http://127.18.39.67:39619/ using document root <none> and password file <none>
I20260619 06:55:36.470180 5809 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:36.470304 5809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:36.470628 5809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:36.473632 5809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "b254a9f698c04aff8004572957723ebc"
format_stamp: "Formatted at 2026-06-19 06:55:36 on dist-test-slave-1h5s"
server_key: "b5e216d647df5b71904778da3820607b"
server_key_iv: "885673cb60b0e6e521227db3e1650c29"
server_key_version: "encryptionkey@0"
I20260619 06:55:36.474413 5809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "b254a9f698c04aff8004572957723ebc"
format_stamp: "Formatted at 2026-06-19 06:55:36 on dist-test-slave-1h5s"
server_key: "b5e216d647df5b71904778da3820607b"
server_key_iv: "885673cb60b0e6e521227db3e1650c29"
server_key_version: "encryptionkey@0"
I20260619 06:55:36.479640 5809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:55:36.483333 5824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:36.484856 5809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:36.485039 5809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "b254a9f698c04aff8004572957723ebc"
format_stamp: "Formatted at 2026-06-19 06:55:36 on dist-test-slave-1h5s"
server_key: "b5e216d647df5b71904778da3820607b"
server_key_iv: "885673cb60b0e6e521227db3e1650c29"
server_key_version: "encryptionkey@0"
I20260619 06:55:36.485262 5809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:36.498728 5809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:36.503082 5809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:36.503427 5809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:36.504531 5809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:36.506253 5809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:36.506357 5809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:36.506490 5809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:36.506568 5809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:36.518486 5669 heartbeater.cc:503] Master 127.18.39.126:44361 was elected leader, sending a full tablet report...
I20260619 06:55:36.541625 5809 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:43835
I20260619 06:55:36.541712 5936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:43835 every 8 connection(s)
I20260619 06:55:36.542938 5809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:36.551087 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5809
I20260619 06:55:36.551290 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:36.551885 18589 external_mini_cluster.cc:1428] Setting key 9fc83cfc6df5715bba6d52f0120a4a51
Jun 19 06:55:36 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852136, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.67@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:55:36.563275 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.545657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:32801 (local address 127.18.39.126:44361)
0619 06:55:36.546183 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.546188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.546948 (+ 760us) server_negotiation.cc:407] Connection header received
0619 06:55:36.548762 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.548768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.548877 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.549009 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:36.550816 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.552042 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.554107 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.554704 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.554830 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.558133 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:36.558138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:36.558141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:36.558172 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:36.560590 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.561277 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.561280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.561283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.561450 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.561837 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.561840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.561842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.562104 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:36.562221 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.562827 (+ 606us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.562988 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":263,"threads_started":1}
I20260619 06:55:36.564246 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.546077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:44361 (local address 127.18.39.67:32801)
0619 06:55:36.546782 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.546824 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.548487 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.549301 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.549312 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.550083 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:36.550629 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.550646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.552200 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.552205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.553949 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.553961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.554929 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.554936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.555162 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.555803 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:55:36.555824 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:36.557929 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:36.560723 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:36.560730 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:36.560743 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:36.561150 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:36.561541 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:55:36.561544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:55:36.561546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:55:36.561737 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:55:36.562211 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:36.562217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:36.562525 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.562741 (+ 216us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.563114 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":271,"threads_started":1}
I20260619 06:55:36.565795 5937 heartbeater.cc:347] Connected to a master server at 127.18.39.126:44361
I20260619 06:55:36.566184 5937 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:36.567183 5937 heartbeater.cc:511] Master 127.18.39.126:44361 requested a full tablet report, sending...
I20260619 06:55:36.568542 5480 ts_manager.cc:194] Registered new tserver with Master: b254a9f698c04aff8004572957723ebc (127.18.39.67:43835)
I20260619 06:55:36.569383 5480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.39.67@KRBTEST.COM'} at 127.18.39.67:32801
I20260619 06:55:36.579597 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:36.596573 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.584168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.18.39.126:44361)
0619 06:55:36.584350 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.584354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.584726 (+ 372us) server_negotiation.cc:407] Connection header received
0619 06:55:36.584844 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.584847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.584912 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.584994 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:36.586442 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.587176 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.588467 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.588888 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.589079 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.591564 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:36.591570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:36.591573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:36.591605 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:36.594162 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.594907 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.594911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.594914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.595034 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.595384 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.595387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.595389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.595715 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:36.595891 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.596179 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.596354 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:55:36.600790 5480 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:36.608258 5480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:36.633256 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.621376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.18.39.67:43835)
0619 06:55:36.622942 (+ 1566us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.622952 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.623364 (+ 412us) server_negotiation.cc:407] Connection header received
0619 06:55:36.623660 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.623668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.623943 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.624121 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:36.625155 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.626872 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.630093 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.631591 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.631702 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.631966 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.632702 (+ 736us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.632890 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1367,"thread_start_us":1203,"threads_started":1}
I20260619 06:55:36.634876 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.620671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:43835 (local address 127.0.0.1:49900)
0619 06:55:36.623255 (+ 2584us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.623272 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.623472 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.624472 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.624479 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.624511 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:36.624946 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.624960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.627062 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.627068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.629898 (+ 2830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.629913 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.632167 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.632174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.632376 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.632440 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.633836 (+ 1396us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.634064 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2461,"mutex_wait_us":77,"thread_start_us":2291,"threads_started":1}
I20260619 06:55:36.635562 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.620426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:45675 (local address 127.0.0.1:58518)
0619 06:55:36.622055 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.622097 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.622272 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.625212 (+ 2940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.625220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.625250 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:36.625692 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.625703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.629092 (+ 3389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.629098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.631355 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.631370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.634615 (+ 3245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.634629 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.634879 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.634914 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.635000 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.635136 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1403,"mutex_wait_us":1,"thread_start_us":1260,"threads_started":1}
I20260619 06:55:36.636025 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.620634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.18.39.66:45675)
0619 06:55:36.623413 (+ 2779us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.623422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.623447 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:55:36.624681 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.624692 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.624999 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.625906 (+ 907us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:36.626125 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.628934 (+ 2809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.632721 (+ 3787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.634269 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.634540 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.634743 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.635482 (+ 739us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.635692 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2624,"thread_start_us":2487,"threads_started":1}
I20260619 06:55:36.639277 5738 tablet_service.cc:1467] Processing CreateTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b (DEFAULT_TABLE table=test-table [id=f5ac2457f9524749b905f16b59121645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:36.639277 5872 tablet_service.cc:1467] Processing CreateTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b (DEFAULT_TABLE table=test-table [id=f5ac2457f9524749b905f16b59121645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:36.640822 5738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d17e094d45d41a8b5c98652fa7cf80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:36.640822 5872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d17e094d45d41a8b5c98652fa7cf80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:36.642446 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.623409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:38327 (local address 127.0.0.1:38880)
0619 06:55:36.635150 (+ 11741us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.635171 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.635300 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.636382 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.636387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.636417 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:36.637188 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.637200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.638689 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.638695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.640092 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.640115 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.641622 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.641629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.641800 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.641839 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.641958 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.642102 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10893,"thread_start_us":11099,"threads_started":1}
I20260619 06:55:36.642481 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.623853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.18.39.65:38327)
0619 06:55:36.635353 (+ 11500us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.635365 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.635394 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:55:36.635499 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.635506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.635970 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.636421 (+ 451us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:36.637396 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.638526 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.640255 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.641408 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.641563 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.641799 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.642012 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.642175 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11320,"thread_start_us":10960,"threads_started":1}
I20260619 06:55:36.646999 5604 tablet_service.cc:1467] Processing CreateTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b (DEFAULT_TABLE table=test-table [id=f5ac2457f9524749b905f16b59121645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:36.648638 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d17e094d45d41a8b5c98652fa7cf80b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:36.655227 5956 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Bootstrap starting.
I20260619 06:55:36.657440 5957 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Bootstrap starting.
I20260619 06:55:36.659201 5956 tablet_bootstrap.cc:654] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:36.660650 5956 log.cc:824] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:36.660928 5957 tablet_bootstrap.cc:654] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:36.662374 5957 log.cc:824] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:36.664026 5956 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: No bootstrap required, opened a new log
I20260619 06:55:36.664108 5960 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Bootstrap starting.
I20260619 06:55:36.664321 5956 ts_tablet_manager.cc:1397] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:55:36.665614 5957 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: No bootstrap required, opened a new log
I20260619 06:55:36.665877 5957 ts_tablet_manager.cc:1397] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 06:55:36.667763 5960 tablet_bootstrap.cc:654] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:36.669001 5960 log.cc:824] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:36.672542 5960 tablet_bootstrap.cc:492] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: No bootstrap required, opened a new log
I20260619 06:55:36.672827 5960 ts_tablet_manager.cc:1397] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:55:36.674454 5957 raft_consensus.cc:348] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.674844 5957 raft_consensus.cc:374] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:36.674942 5957 raft_consensus.cc:729] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73b25ff5f95e4af483c4fb02388e5ad0, State: Initialized, Role: FOLLOWER
I20260619 06:55:36.674881 5956 raft_consensus.cc:348] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.675292 5956 raft_consensus.cc:374] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:36.675375 5956 raft_consensus.cc:729] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b254a9f698c04aff8004572957723ebc, State: Initialized, Role: FOLLOWER
I20260619 06:55:36.675590 5957 consensus_queue.cc:260] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.675966 5956 consensus_queue.cc:260] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.676976 5803 heartbeater.cc:503] Master 127.18.39.126:44361 was elected leader, sending a full tablet report...
I20260619 06:55:36.677517 5957 ts_tablet_manager.cc:1428] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260619 06:55:36.679841 5937 heartbeater.cc:503] Master 127.18.39.126:44361 was elected leader, sending a full tablet report...
I20260619 06:55:36.680440 5956 ts_tablet_manager.cc:1428] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260619 06:55:36.685587 5960 raft_consensus.cc:348] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.685971 5960 raft_consensus.cc:374] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:36.686046 5960 raft_consensus.cc:729] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a48f9cf7cb3440bbd7dff473649cd0e, State: Initialized, Role: FOLLOWER
I20260619 06:55:36.686648 5960 consensus_queue.cc:260] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.688040 5960 ts_tablet_manager.cc:1428] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260619 06:55:36.719720 5963 raft_consensus.cc:482] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:36.720095 5963 raft_consensus.cc:504] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.721948 5963 leader_election.cc:288] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73b25ff5f95e4af483c4fb02388e5ad0 (127.18.39.66:45675), 2a48f9cf7cb3440bbd7dff473649cd0e (127.18.39.65:38327)
I20260619 06:55:36.727798 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.722380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:60695 (local address 127.18.39.65:38327)
0619 06:55:36.722619 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.722627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.722801 (+ 174us) server_negotiation.cc:407] Connection header received
0619 06:55:36.723043 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.723048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.723123 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.723218 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:36.723600 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.724413 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.726161 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.726930 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.727098 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.727185 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.727428 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.727553 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:55:36.727844 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.722345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:38327 (local address 127.18.39.67:60695)
0619 06:55:36.722678 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.722701 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.722886 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.723249 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.723253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.723279 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:36.723481 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.723494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.724537 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.724545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.726004 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.726016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.727083 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.727089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.727269 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.727292 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.727400 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.727534 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":50}
I20260619 06:55:36.728507 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.722471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:53111 (local address 127.18.39.66:45675)
0619 06:55:36.722667 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.722672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.723250 (+ 578us) server_negotiation.cc:407] Connection header received
0619 06:55:36.723522 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.723530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.723605 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.723762 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:36.724320 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.725073 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.726782 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.727555 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.727740 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.727807 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.728133 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.728273 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:36.728585 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.722385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:45675 (local address 127.18.39.67:53111)
0619 06:55:36.723131 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:36.723148 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:36.723343 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:36.723777 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:36.723781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:36.723804 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:36.724160 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.724170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.725217 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.725222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.726611 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:36.726622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.727760 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.727767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:36.727959 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.728006 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 06:55:36.728091 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 06:55:36.728220 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":20,"spinlock_wait_cycles":24448,"thread_start_us":418,"threads_started":1}
I20260619 06:55:36.728936 5624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d17e094d45d41a8b5c98652fa7cf80b" candidate_uuid: "b254a9f698c04aff8004572957723ebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" is_pre_election: true
I20260619 06:55:36.729513 5624 raft_consensus.cc:2454] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b254a9f698c04aff8004572957723ebc in term 0.
I20260619 06:55:36.729768 5758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d17e094d45d41a8b5c98652fa7cf80b" candidate_uuid: "b254a9f698c04aff8004572957723ebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" is_pre_election: true
I20260619 06:55:36.730224 5828 leader_election.cc:302] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a48f9cf7cb3440bbd7dff473649cd0e, b254a9f698c04aff8004572957723ebc; no voters:
I20260619 06:55:36.730260 5758 raft_consensus.cc:2454] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b254a9f698c04aff8004572957723ebc in term 0.
I20260619 06:55:36.730759 5963 raft_consensus.cc:2781] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:36.730954 5963 raft_consensus.cc:482] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:36.731093 5963 raft_consensus.cc:3037] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:36.733400 5963 raft_consensus.cc:504] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.734006 5963 leader_election.cc:288] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [CANDIDATE]: Term 1 election: Requested vote from peers 73b25ff5f95e4af483c4fb02388e5ad0 (127.18.39.66:45675), 2a48f9cf7cb3440bbd7dff473649cd0e (127.18.39.65:38327)
I20260619 06:55:36.734545 5758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d17e094d45d41a8b5c98652fa7cf80b" candidate_uuid: "b254a9f698c04aff8004572957723ebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73b25ff5f95e4af483c4fb02388e5ad0"
I20260619 06:55:36.734546 5624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d17e094d45d41a8b5c98652fa7cf80b" candidate_uuid: "b254a9f698c04aff8004572957723ebc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e"
I20260619 06:55:36.734773 5758 raft_consensus.cc:3037] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:36.734788 5624 raft_consensus.cc:3037] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 0 FOLLOWER]: Advancing to term 1
W20260619 06:55:36.737471 5670 tablet.cc:2369] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:36.738144 5758 raft_consensus.cc:2454] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b254a9f698c04aff8004572957723ebc in term 1.
I20260619 06:55:36.738153 5624 raft_consensus.cc:2454] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b254a9f698c04aff8004572957723ebc in term 1.
I20260619 06:55:36.738581 5828 leader_election.cc:302] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a48f9cf7cb3440bbd7dff473649cd0e, b254a9f698c04aff8004572957723ebc; no voters:
I20260619 06:55:36.738941 5963 raft_consensus.cc:2781] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:36.739974 5963 raft_consensus.cc:686] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 1 LEADER]: Becoming Leader. State: Replica: b254a9f698c04aff8004572957723ebc, State: Running, Role: LEADER
I20260619 06:55:36.740585 5963 consensus_queue.cc:237] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } }
I20260619 06:55:36.746025 5479 catalog_manager.cc:5495] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc reported cstate change: term changed from 0 to 1, leader changed from <none> to b254a9f698c04aff8004572957723ebc (127.18.39.67). New cstate: current_term: 1 leader_uuid: "b254a9f698c04aff8004572957723ebc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b254a9f698c04aff8004572957723ebc" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 43835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 } health_report { overall_health: UNKNOWN } } }
W20260619 06:55:36.758635 5804 tablet.cc:2369] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:36.787235 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.781610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.18.39.67:43835)
0619 06:55:36.781881 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.781892 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.781911 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:55:36.782192 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.782197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.782281 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.782415 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:36.782740 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.783469 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.784858 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.785400 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.785531 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.786064 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:36.786639 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:36.786782 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.786911 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.786988 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260619 06:55:36.794582 5938 tablet.cc:2369] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:36.796106 5758 raft_consensus.cc:1261] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 1 FOLLOWER]: Refusing update from remote peer b254a9f698c04aff8004572957723ebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:36.796095 5624 raft_consensus.cc:1261] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 1 FOLLOWER]: Refusing update from remote peer b254a9f698c04aff8004572957723ebc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:36.797251 5966 consensus_queue.cc:1035] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a48f9cf7cb3440bbd7dff473649cd0e" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:36.797681 5963 consensus_queue.cc:1035] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73b25ff5f95e4af483c4fb02388e5ad0" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 45675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:36.810923 5969 mvcc.cc:205] Tried to move back new op lower bound from 7298466352306331648 to 7298466352094629888. Current Snapshot: MvccSnapshot[applied={T|T < 7298466352306331648}]
I20260619 06:55:36.811056 5971 mvcc.cc:205] Tried to move back new op lower bound from 7298466352306331648 to 7298466352094629888. Current Snapshot: MvccSnapshot[applied={T|T < 7298466352306331648}]
I20260619 06:55:36.825165 5480 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:36.825505 5480 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:36.829516 5480 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e81077f6594d4783966f5f37577cb7f0: Sending DeleteTablet for 3 replicas of tablet 2d17e094d45d41a8b5c98652fa7cf80b
I20260619 06:55:36.830830 5738 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:36 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:58518
I20260619 06:55:36.830983 5604 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:36 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:38880
I20260619 06:55:36.831801 5872 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d17e094d45d41a8b5c98652fa7cf80b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:36 UTC) from {username='kudu', principal='kudu/127.18.39.126@KRBTEST.COM'} at 127.0.0.1:49900
I20260619 06:55:36.831983 5979 tablet_replica.cc:330] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: stopping tablet replica
I20260619 06:55:36.832135 5978 tablet_replica.cc:330] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: stopping tablet replica
I20260619 06:55:36.832497 5979 raft_consensus.cc:2229] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:36.832497 5978 raft_consensus.cc:2229] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:36.833062 5979 raft_consensus.cc:2258] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:36.833062 5978 raft_consensus.cc:2258] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:36.833482 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5541
I20260619 06:55:36.835403 5978 ts_tablet_manager.cc:1905] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:36.835764 5979 ts_tablet_manager.cc:1905] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:36.842911 5979 ts_tablet_manager.cc:1918] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:36.843029 5978 ts_tablet_manager.cc:1918] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:36.843209 5978 log.cc:1196] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/2d17e094d45d41a8b5c98652fa7cf80b
I20260619 06:55:36.843739 5980 tablet_replica.cc:330] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: stopping tablet replica
I20260619 06:55:36.843819 5978 ts_tablet_manager.cc:1939] T 2d17e094d45d41a8b5c98652fa7cf80b P 73b25ff5f95e4af483c4fb02388e5ad0: Deleting consensus metadata
I20260619 06:55:36.844364 5980 raft_consensus.cc:2229] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:36.845108 5980 raft_consensus.cc:2258] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:36.846009 5466 catalog_manager.cc:4841] TS 73b25ff5f95e4af483c4fb02388e5ad0 (127.18.39.66:45675): tablet 2d17e094d45d41a8b5c98652fa7cf80b (table test-table [id=f5ac2457f9524749b905f16b59121645]) successfully deleted
I20260619 06:55:36.847200 5980 ts_tablet_manager.cc:1905] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:36.848824 5979 log.cc:1196] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/2d17e094d45d41a8b5c98652fa7cf80b
I20260619 06:55:36.849524 5979 ts_tablet_manager.cc:1939] T 2d17e094d45d41a8b5c98652fa7cf80b P 2a48f9cf7cb3440bbd7dff473649cd0e: Deleting consensus metadata
I20260619 06:55:36.851540 5467 catalog_manager.cc:4841] TS 2a48f9cf7cb3440bbd7dff473649cd0e (127.18.39.65:38327): tablet 2d17e094d45d41a8b5c98652fa7cf80b (table test-table [id=f5ac2457f9524749b905f16b59121645]) successfully deleted
I20260619 06:55:36.854554 5980 ts_tablet_manager.cc:1918] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:36.854750 5980 log.cc:1196] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/2d17e094d45d41a8b5c98652fa7cf80b
I20260619 06:55:36.855332 5980 ts_tablet_manager.cc:1939] T 2d17e094d45d41a8b5c98652fa7cf80b P b254a9f698c04aff8004572957723ebc: Deleting consensus metadata
I20260619 06:55:36.857012 5467 catalog_manager.cc:4841] TS b254a9f698c04aff8004572957723ebc (127.18.39.67:43835): tablet 2d17e094d45d41a8b5c98652fa7cf80b (table test-table [id=f5ac2457f9524749b905f16b59121645]) successfully deleted
Jun 19 06:55:36 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.65@KRBTEST.COM
I20260619 06:55:36.870366 5955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.836207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.18.39.65:38327)
0619 06:55:36.836387 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.836393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.849278 (+ 12885us) server_negotiation.cc:407] Connection header received
0619 06:55:36.849806 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.849814 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.849913 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.849990 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:36.852070 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.853235 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.859002 (+ 5767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.859445 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.859609 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.863937 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:36.863954 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:36.863969 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:36.864022 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:36.867350 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.868110 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.868117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.868120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.868272 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.868681 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.868688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.868696 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.869058 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:36.869412 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.869759 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.869988 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:55:36.871058 5664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:36.962642 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5541
I20260619 06:55:36.981592 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5675
Jun 19 06:55:36 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.66@KRBTEST.COM
I20260619 06:55:36.997701 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:36.982473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.18.39.66:45675)
0619 06:55:36.982691 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:36.982698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:36.982861 (+ 163us) server_negotiation.cc:407] Connection header received
0619 06:55:36.983074 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:36.983080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:36.983169 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:36.983278 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:36.984551 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.985319 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.986617 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:36.987022 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:36.987154 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:36.991613 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:36.991631 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:36.991647 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:36.991688 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:36.994725 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.995430 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.995437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.995439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.995564 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:36.995948 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:36.995955 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:36.995958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:36.996364 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:36.996672 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:36.996977 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 06:55:36.997212 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:55:36.998526 5798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:37.090514 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5675
I20260619 06:55:37.116564 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5809
Jun 19 06:55:37 dist-test-slave-1h5s krb5kdc[5435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.67@KRBTEST.COM
I20260619 06:55:37.136134 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:37.117409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.18.39.67:43835)
0619 06:55:37.117623 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:37.117635 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:37.117668 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:55:37.117857 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:37.117865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:37.117985 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:37.118150 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:37.119751 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:37.121143 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:37.122563 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:37.123370 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:37.123519 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:37.128078 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:37.128095 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:37.128125 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:37.128181 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:37.132726 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:37.133649 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:37.133660 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:37.133664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:37.133863 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:37.134259 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:37.134265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:37.134269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:37.134772 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:37.135136 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:37.135387 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 06:55:37.135664 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:55:37.136873 5932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:37.261816 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5809
I20260619 06:55:37.277134 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5448
I20260619 06:55:37.278317 5510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:37.391197 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5448
2026-06-19T06:55:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4361 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T06:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:37Z Disabled control of system clock
I20260619 06:55:37.468777 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40901
--webserver_interface=127.18.39.126
--webserver_port=0
--builtin_ntp_servers=127.18.39.84:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:40901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:37.805838 5989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:37.806154 5989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:37.806238 5989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:37.813616 5989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:37.813737 5989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:37.813776 5989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:37.813838 5989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:37.813882 5989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:37.823212 5989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:40901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:40901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:37.824504 5989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:37.826459 5989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:37.838406 5995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:37.838332 5994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:37.838436 5997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:37.839082 5989 server_base.cc:1034] running on GCE node
I20260619 06:55:37.839947 5989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:37.841462 5989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:37.842691 5989 hybrid_clock.cc:648] HybridClock initialized: now 1781852137842644 us; error 56 us; skew 500 ppm
I20260619 06:55:37.843065 5989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:37.849491 5989 webserver.cc:460] Webserver started at http://127.18.39.126:41243/ using document root <none> and password file <none>
I20260619 06:55:37.850302 5989 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:37.850453 5989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:37.850792 5989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:37.854013 5989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "6f9eb0cfadb942c3ad120aebd071b4eb"
format_stamp: "Formatted at 2026-06-19 06:55:37 on dist-test-slave-1h5s"
server_key: "42f4b3383c09e440b16a1ba9ba9ab02d"
server_key_iv: "967f66674a16c7a01df90a615046f61c"
server_key_version: "encryptionkey@0"
I20260619 06:55:37.854939 5989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "6f9eb0cfadb942c3ad120aebd071b4eb"
format_stamp: "Formatted at 2026-06-19 06:55:37 on dist-test-slave-1h5s"
server_key: "42f4b3383c09e440b16a1ba9ba9ab02d"
server_key_iv: "967f66674a16c7a01df90a615046f61c"
server_key_version: "encryptionkey@0"
I20260619 06:55:37.860508 5989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:55:37.864388 6003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:37.866081 5989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:55:37.866317 5989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "6f9eb0cfadb942c3ad120aebd071b4eb"
format_stamp: "Formatted at 2026-06-19 06:55:37 on dist-test-slave-1h5s"
server_key: "42f4b3383c09e440b16a1ba9ba9ab02d"
server_key_iv: "967f66674a16c7a01df90a615046f61c"
server_key_version: "encryptionkey@0"
I20260619 06:55:37.866570 5989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:37.889611 5989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:37.890542 5989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:37.890870 5989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:37.910636 5989 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:40901
I20260619 06:55:37.910653 6054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:40901 every 8 connection(s)
I20260619 06:55:37.912249 5989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:37.914113 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 5989
I20260619 06:55:37.914315 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:37.914954 18589 external_mini_cluster.cc:1428] Setting key 68de99121623ce6a9b40318390b09a07
I20260619 06:55:37.917990 6055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:37.930371 6058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:37.916950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.18.39.126:40901)
0619 06:55:37.918082 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:37.918095 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:37.918182 (+ 87us) server_negotiation.cc:407] Connection header received
0619 06:55:37.920030 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:37.920054 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:37.920575 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:37.921002 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:37.921735 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:37.923112 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:37.925204 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:37.926021 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:37.926167 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:37.927588 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:37.927597 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:37.927652 (+ 55us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:37.928295 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:37.928381 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:37.928408 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:37.928523 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:37.928812 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 06:55:37.929249 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":443,"threads_started":1}
I20260619 06:55:37.931162 6055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: Bootstrap starting.
I20260619 06:55:37.934861 6055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:37.935882 6055 log.cc:824] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:37.939073 6055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: No bootstrap required, opened a new log
I20260619 06:55:37.946973 6055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } }
I20260619 06:55:37.947363 6055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:37.947439 6055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9eb0cfadb942c3ad120aebd071b4eb, State: Initialized, Role: FOLLOWER
I20260619 06:55:37.948040 6055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } }
I20260619 06:55:37.948251 6055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:37.948354 6055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:37.948480 6055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:37.950764 6055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } }
I20260619 06:55:37.951242 6055 leader_election.cc:302] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f9eb0cfadb942c3ad120aebd071b4eb; no voters:
I20260619 06:55:37.951828 6055 leader_election.cc:288] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:37.952015 6060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:37.953325 6060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [term 1 LEADER]: Becoming Leader. State: Replica: 6f9eb0cfadb942c3ad120aebd071b4eb, State: Running, Role: LEADER
I20260619 06:55:37.954006 6060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } }
I20260619 06:55:37.954514 6055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:37.957082 6061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } } }
I20260619 06:55:37.957060 6062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f9eb0cfadb942c3ad120aebd071b4eb. Latest consensus state: current_term: 1 leader_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f9eb0cfadb942c3ad120aebd071b4eb" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 40901 } } }
I20260619 06:55:37.957603 6061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:37.957614 6062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:37.959579 6066 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:37.964538 6066 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:37.973701 6066 catalog_manager.cc:1347] Generated new cluster ID: e77ae3c8a084488d9d30c19fbb706007
I20260619 06:55:37.973846 6066 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:37.995345 6066 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:37.996747 6066 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:38.006119 6066 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: Generated new TSK 0
I20260619 06:55:38.007068 6066 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:38.018508 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.65:0
--local_ip_for_outbound_sockets=127.18.39.65
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--builtin_ntp_servers=127.18.39.84:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:55:38.363157 6079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:38.363519 6079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:38.363687 6079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:38.371760 6079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:38.371917 6079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:38.372144 6079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.65
I20260619 06:55:38.381837 6079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.39.65
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.65
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:38.383224 6079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:38.385047 6079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:38.398893 6087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:38.398854 6084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:38.398942 6085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:38.402925 6079 server_base.cc:1034] running on GCE node
I20260619 06:55:38.403595 6079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:38.404639 6079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:38.405856 6079 hybrid_clock.cc:648] HybridClock initialized: now 1781852138405808 us; error 38 us; skew 500 ppm
I20260619 06:55:38.406203 6079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:38.408480 6079 webserver.cc:460] Webserver started at http://127.18.39.65:40939/ using document root <none> and password file <none>
I20260619 06:55:38.409195 6079 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:38.409312 6079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:38.409669 6079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:38.412572 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data/instance:
uuid: "a13b5fc029c74a9ab33a589406acce88"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "1d7bccc99cc80e71a61d45f6e5c7baa9"
server_key_iv: "b3513a694a90c19755b15aff36980377"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.413458 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance:
uuid: "a13b5fc029c74a9ab33a589406acce88"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "1d7bccc99cc80e71a61d45f6e5c7baa9"
server_key_iv: "b3513a694a90c19755b15aff36980377"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.418638 6079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260619 06:55:38.422185 6093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.423722 6079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:38.423872 6079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal
uuid: "a13b5fc029c74a9ab33a589406acce88"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "1d7bccc99cc80e71a61d45f6e5c7baa9"
server_key_iv: "b3513a694a90c19755b15aff36980377"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.424070 6079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:38.438714 6079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:38.439531 6079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:38.439790 6079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:38.440714 6079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:38.442255 6079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:38.442343 6079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.442422 6079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:38.442478 6079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.476341 6079 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.65:43663
I20260619 06:55:38.476390 6205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.65:43663 every 8 connection(s)
I20260619 06:55:38.477804 6079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/data/info.pb
I20260619 06:55:38.484320 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6079
I20260619 06:55:38.484519 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal/instance
I20260619 06:55:38.485100 18589 external_mini_cluster.cc:1428] Setting key 3751e6e3b6e2245b8c376fdccfed9083
I20260619 06:55:38.488117 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.66:0
--local_ip_for_outbound_sockets=127.18.39.66
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--builtin_ntp_servers=127.18.39.84:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:38.491946 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:38.480497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:33779 (local address 127.18.39.126:40901)
0619 06:55:38.481032 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:38.481037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:38.482351 (+ 1314us) server_negotiation.cc:407] Connection header received
0619 06:55:38.484426 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:38.484433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:38.484545 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:38.484686 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:38.486256 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.487003 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.488792 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.489256 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.489456 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:38.490979 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:38.490983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:38.490987 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:38.491007 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:38.491048 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:38.491060 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:38.491231 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:38.491481 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 06:55:38.491590 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":296,"threads_started":1}
I20260619 06:55:38.493075 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:38.481135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40901 (local address 127.18.39.65:33779)
0619 06:55:38.482160 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:38.482210 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:38.484086 (+ 1876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:38.485040 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:38.485053 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:38.485194 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:38.486059 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:38.486079 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.487183 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.487187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:38.488616 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:38.488631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.489521 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.489530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:38.489801 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:38.490567 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:38.490577 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:38.490624 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:38.490759 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:38.491237 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:38.491252 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:38.491262 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:55:38.491455 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 06:55:38.491916 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":333,"threads_started":1}
I20260619 06:55:38.494846 6206 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40901
I20260619 06:55:38.495179 6206 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:38.495970 6206 heartbeater.cc:511] Master 127.18.39.126:40901 requested a full tablet report, sending...
I20260619 06:55:38.498205 6020 ts_manager.cc:194] Registered new tserver with Master: a13b5fc029c74a9ab33a589406acce88 (127.18.39.65:43663)
I20260619 06:55:38.499768 6020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.65:33779
W20260619 06:55:38.830600 6210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:38.830946 6210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:38.831110 6210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:38.838584 6210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:38.838752 6210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:38.838945 6210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.66
I20260619 06:55:38.848317 6210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.39.66
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.66
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:38.849725 6210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:38.851526 6210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:38.866890 6215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:38.866851 6216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:38.867234 6218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:38.868583 6210 server_base.cc:1034] running on GCE node
I20260619 06:55:38.869506 6210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:38.870731 6210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:38.871980 6210 hybrid_clock.cc:648] HybridClock initialized: now 1781852138871915 us; error 58 us; skew 500 ppm
I20260619 06:55:38.872328 6210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:38.874923 6210 webserver.cc:460] Webserver started at http://127.18.39.66:46863/ using document root <none> and password file <none>
I20260619 06:55:38.875672 6210 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:38.875798 6210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:38.876114 6210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:38.879225 6210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data/instance:
uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "14406ce5085568c764e6d29b23f7bc83"
server_key_iv: "f42904b5dbf8f33c80eb5ddafb5426ff"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.880062 6210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance:
uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "14406ce5085568c764e6d29b23f7bc83"
server_key_iv: "f42904b5dbf8f33c80eb5ddafb5426ff"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.885560 6210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:55:38.889245 6224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.890636 6210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:55:38.890865 6210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal
uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b"
format_stamp: "Formatted at 2026-06-19 06:55:38 on dist-test-slave-1h5s"
server_key: "14406ce5085568c764e6d29b23f7bc83"
server_key_iv: "f42904b5dbf8f33c80eb5ddafb5426ff"
server_key_version: "encryptionkey@0"
I20260619 06:55:38.891093 6210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:38.913172 6210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:38.914172 6210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:38.914494 6210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:38.915515 6210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:38.917184 6210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:38.917313 6210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.917472 6210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:38.917560 6210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:38.951362 6210 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.66:33329
I20260619 06:55:38.951452 6336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.66:33329 every 8 connection(s)
I20260619 06:55:38.952750 6210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/data/info.pb
I20260619 06:55:38.956634 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6210
I20260619 06:55:38.956866 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal/instance
I20260619 06:55:38.957401 18589 external_mini_cluster.cc:1428] Setting key 3e6a46cf227f42ed4eccf8b109dd96a9
I20260619 06:55:38.960297 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.39.67:0
--local_ip_for_outbound_sockets=127.18.39.67
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--builtin_ntp_servers=127.18.39.84:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:55:38.966090 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:38.955513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.66:38383 (local address 127.18.39.126:40901)
0619 06:55:38.955756 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:38.955761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:38.957383 (+ 1622us) server_negotiation.cc:407] Connection header received
0619 06:55:38.959460 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:38.959465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:38.959556 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:38.959674 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:38.960876 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.961763 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.963319 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.963733 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.963868 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:38.965253 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:38.965258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:38.965261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:38.965281 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:38.965314 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:38.965327 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:38.965454 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:38.965676 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 06:55:38.965777 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:55:38.967198 6339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:38.956112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40901 (local address 127.18.39.66:38383)
0619 06:55:38.957183 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:38.957239 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:38.959182 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:38.959916 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:38.959928 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:38.960032 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:38.960719 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:38.960737 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.961903 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.961909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:38.963172 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:38.963183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:38.963950 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:38.963956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:38.964195 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:38.964918 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:38.964920 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:38.964960 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:38.965042 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:38.965451 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:38.965462 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:38.965472 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:55:38.965659 (+ 187us) client_negotiation.cc:240] Negotiation successful
0619 06:55:38.966054 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":330,"threads_started":1}
I20260619 06:55:38.968763 6337 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40901
I20260619 06:55:38.969094 6337 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:38.969955 6337 heartbeater.cc:511] Master 127.18.39.126:40901 requested a full tablet report, sending...
I20260619 06:55:38.971794 6020 ts_manager.cc:194] Registered new tserver with Master: c0613d6b18aa4d4ca2c2e1c56dbd601b (127.18.39.66:33329)
I20260619 06:55:38.972656 6020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.66:38383
W20260619 06:55:39.294849 6340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:39.295195 6340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:39.295357 6340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:39.302847 6340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:39.303020 6340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:39.303207 6340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.39.67
I20260619 06:55:39.312630 6340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.39.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.39.67
--webserver_port=0
--tserver_master_addrs=127.18.39.126:40901
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.39.67
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:39.314054 6340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:39.315938 6340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:39.329726 6345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:39.329689 6348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:39.329705 6346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:39.330600 6340 server_base.cc:1034] running on GCE node
I20260619 06:55:39.331247 6340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:39.332286 6340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:39.333499 6340 hybrid_clock.cc:648] HybridClock initialized: now 1781852139333437 us; error 60 us; skew 500 ppm
I20260619 06:55:39.333834 6340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:55:39.336131 6340 webserver.cc:460] Webserver started at http://127.18.39.67:44379/ using document root <none> and password file <none>
I20260619 06:55:39.336894 6340 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:39.337024 6340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:39.337392 6340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:39.340399 6340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data/instance:
uuid: "27d7bb5cdd2243248de14d46777fc64e"
format_stamp: "Formatted at 2026-06-19 06:55:39 on dist-test-slave-1h5s"
server_key: "6787be4fba187f95e50e49ca2fc2d15e"
server_key_iv: "e042f6e6dc3d872a5648db463d0b628c"
server_key_version: "encryptionkey@0"
I20260619 06:55:39.341439 6340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance:
uuid: "27d7bb5cdd2243248de14d46777fc64e"
format_stamp: "Formatted at 2026-06-19 06:55:39 on dist-test-slave-1h5s"
server_key: "6787be4fba187f95e50e49ca2fc2d15e"
server_key_iv: "e042f6e6dc3d872a5648db463d0b628c"
server_key_version: "encryptionkey@0"
I20260619 06:55:39.346798 6340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 06:55:39.350428 6354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:39.351771 6340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:55:39.351966 6340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal
uuid: "27d7bb5cdd2243248de14d46777fc64e"
format_stamp: "Formatted at 2026-06-19 06:55:39 on dist-test-slave-1h5s"
server_key: "6787be4fba187f95e50e49ca2fc2d15e"
server_key_iv: "e042f6e6dc3d872a5648db463d0b628c"
server_key_version: "encryptionkey@0"
I20260619 06:55:39.352195 6340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:39.364682 6340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:39.365669 6340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:39.365984 6340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:39.367013 6340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:55:39.368539 6340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:55:39.368686 6340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:39.368819 6340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:55:39.368908 6340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:39.403893 6340 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.67:33233
I20260619 06:55:39.403929 6466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.67:33233 every 8 connection(s)
I20260619 06:55:39.405318 6340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/data/info.pb
I20260619 06:55:39.406833 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6340
I20260619 06:55:39.407024 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal/instance
I20260619 06:55:39.407517 18589 external_mini_cluster.cc:1428] Setting key 4dad9465903255bfcf2463e005e8fb74
I20260619 06:55:39.418540 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.408249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.67:35325 (local address 127.18.39.126:40901)
0619 06:55:39.408453 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.408459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.410270 (+ 1811us) server_negotiation.cc:407] Connection header received
0619 06:55:39.412134 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.412138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.412212 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.412299 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:39.413466 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.414220 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.415839 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.416239 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.416334 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.417793 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:39.417798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:39.417801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:39.417829 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:39.417860 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:39.417872 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:39.417950 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.418221 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.418301 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:39.419641 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.408966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:40901 (local address 127.18.39.67:35325)
0619 06:55:39.410099 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.410155 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.411867 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.412520 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.412528 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.412637 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:55:39.413253 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.413269 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.414440 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.414444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.415702 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.415714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.416424 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.416429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.416615 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.417439 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:55:39.417442 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:55:39.417485 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:55:39.417562 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:55:39.417972 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:55:39.417988 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:55:39.417994 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.418149 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.418451 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":380,"threads_started":1}
I20260619 06:55:39.421214 6467 heartbeater.cc:347] Connected to a master server at 127.18.39.126:40901
I20260619 06:55:39.421625 6467 heartbeater.cc:464] Registering TS with master...
I20260619 06:55:39.422535 6467 heartbeater.cc:511] Master 127.18.39.126:40901 requested a full tablet report, sending...
I20260619 06:55:39.424012 6020 ts_manager.cc:194] Registered new tserver with Master: 27d7bb5cdd2243248de14d46777fc64e (127.18.39.67:33233)
I20260619 06:55:39.424822 6020 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.39.67:35325
I20260619 06:55:39.435474 18589 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:55:39.445206 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.439745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.18.39.126:40901)
0619 06:55:39.439975 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.439982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.440244 (+ 262us) server_negotiation.cc:407] Connection header received
0619 06:55:39.440383 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.440387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.440464 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.440544 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:39.440991 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.441764 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.443080 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.443464 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.443553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.444517 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:39.444521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:39.444525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:39.444542 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:39.444573 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:39.444584 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:39.444649 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.444773 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.444874 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:55:39.448951 6020 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:55:39.456208 6020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:55:39.493798 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.469579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55046 (local address 127.18.39.66:33329)
0619 06:55:39.480802 (+ 11223us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.480816 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.484280 (+ 3464us) server_negotiation.cc:407] Connection header received
0619 06:55:39.484940 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.484948 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.485174 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.485391 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:39.486735 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.488372 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.491023 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.492581 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.492791 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.493064 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.493250 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.493444 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10929,"thread_start_us":11250,"threads_started":1}
I20260619 06:55:39.493841 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.469401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:33329 (local address 127.0.0.1:55046)
0619 06:55:39.484141 (+ 14740us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.484159 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.484698 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.485752 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.485763 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.485809 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:39.486385 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.486403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.488546 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.488553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.490822 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.490835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.492822 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.492828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.493012 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.493061 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.493195 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.493418 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14614,"mutex_wait_us":166,"thread_start_us":12429,"threads_started":1}
I20260619 06:55:39.494800 6483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.483439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:33233 (local address 127.0.0.1:57154)
0619 06:55:39.485876 (+ 2437us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.485899 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.486021 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.486881 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.486885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.486910 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:39.487309 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.487321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.489237 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.489247 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.491581 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.491593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.493900 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.493906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.494105 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.494160 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.494251 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.494450 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2255,"mutex_wait_us":2,"thread_start_us":2057,"threads_started":1}
I20260619 06:55:39.494992 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.469195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47812 (local address 127.18.39.65:43663)
0619 06:55:39.481609 (+ 12414us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.481621 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.482788 (+ 1167us) server_negotiation.cc:407] Connection header received
0619 06:55:39.484677 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.484686 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.484919 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.485104 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:39.487028 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.488502 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.491605 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.493222 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.493426 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.493689 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.494416 (+ 727us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.494578 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12173,"thread_start_us":13205,"threads_started":1}
I20260619 06:55:39.495663 6481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.469028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.65:43663 (local address 127.0.0.1:47812)
0619 06:55:39.482531 (+ 13503us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.482600 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.484455 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.486087 (+ 1632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.486093 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.486122 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:39.486784 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.486803 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.488682 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.488686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.491361 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.491376 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.493887 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.493894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.494106 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.494242 (+ 136us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.495173 (+ 931us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.495318 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13088,"mutex_wait_us":322,"thread_start_us":14873,"threads_started":1}
I20260619 06:55:39.495810 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.483635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.18.39.67:33233)
0619 06:55:39.486168 (+ 2533us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.486187 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.486340 (+ 153us) server_negotiation.cc:407] Connection header received
0619 06:55:39.486458 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.486467 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.486680 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.486885 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:39.487520 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.489063 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.491793 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.493315 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.493450 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.493702 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.495200 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.495410 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2354,"thread_start_us":2197,"threads_started":1}
I20260619 06:55:39.498448 6272 tablet_service.cc:1467] Processing CreateTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 (DEFAULT_TABLE table=test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:39.499675 6272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fdaed708c449a6999f22ae9ad5ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:39.499238 6141 tablet_service.cc:1467] Processing CreateTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 (DEFAULT_TABLE table=test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:39.499238 6402 tablet_service.cc:1467] Processing CreateTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 (DEFAULT_TABLE table=test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:55:39.500615 6402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fdaed708c449a6999f22ae9ad5ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:39.500619 6141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20fdaed708c449a6999f22ae9ad5ca45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:39.502910 6206 heartbeater.cc:503] Master 127.18.39.126:40901 was elected leader, sending a full tablet report...
I20260619 06:55:39.512203 6485 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Bootstrap starting.
I20260619 06:55:39.514863 6486 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Bootstrap starting.
I20260619 06:55:39.516144 6485 tablet_bootstrap.cc:654] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:39.518090 6485 log.cc:824] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:39.518893 6486 tablet_bootstrap.cc:654] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:39.519723 6487 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Bootstrap starting.
I20260619 06:55:39.520254 6486 log.cc:824] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:39.521456 6485 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: No bootstrap required, opened a new log
I20260619 06:55:39.521775 6485 ts_tablet_manager.cc:1397] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 06:55:39.523589 6486 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: No bootstrap required, opened a new log
I20260619 06:55:39.523699 6487 tablet_bootstrap.cc:654] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:39.523856 6486 ts_tablet_manager.cc:1397] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:55:39.524978 6487 log.cc:824] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:39.533004 6485 raft_consensus.cc:348] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.533584 6485 raft_consensus.cc:374] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:39.533725 6485 raft_consensus.cc:729] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0613d6b18aa4d4ca2c2e1c56dbd601b, State: Initialized, Role: FOLLOWER
I20260619 06:55:39.534547 6487 tablet_bootstrap.cc:492] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: No bootstrap required, opened a new log
I20260619 06:55:39.534417 6485 consensus_queue.cc:260] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.534874 6487 ts_tablet_manager.cc:1397] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.004s
I20260619 06:55:39.534754 6486 raft_consensus.cc:348] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.535389 6486 raft_consensus.cc:374] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:39.535567 6486 raft_consensus.cc:729] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27d7bb5cdd2243248de14d46777fc64e, State: Initialized, Role: FOLLOWER
I20260619 06:55:39.536441 6486 consensus_queue.cc:260] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.537403 6337 heartbeater.cc:503] Master 127.18.39.126:40901 was elected leader, sending a full tablet report...
I20260619 06:55:39.540359 6485 ts_tablet_manager.cc:1428] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Time spent starting tablet: real 0.018s user 0.011s sys 0.004s
I20260619 06:55:39.541527 6467 heartbeater.cc:503] Master 127.18.39.126:40901 was elected leader, sending a full tablet report...
I20260619 06:55:39.542217 6486 ts_tablet_manager.cc:1428] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260619 06:55:39.544946 6487 raft_consensus.cc:348] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.545531 6487 raft_consensus.cc:374] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:39.545650 6487 raft_consensus.cc:729] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13b5fc029c74a9ab33a589406acce88, State: Initialized, Role: FOLLOWER
I20260619 06:55:39.546365 6487 consensus_queue.cc:260] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.547884 6487 ts_tablet_manager.cc:1428] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
W20260619 06:55:39.657227 6468 tablet.cc:2369] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:39.668021 6493 raft_consensus.cc:482] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:55:39.668391 6493 raft_consensus.cc:504] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.670249 6493 leader_election.cc:288] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0613d6b18aa4d4ca2c2e1c56dbd601b (127.18.39.66:33329), 27d7bb5cdd2243248de14d46777fc64e (127.18.39.67:33233)
I20260619 06:55:39.676892 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.670842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:53225 (local address 127.18.39.66:33329)
0619 06:55:39.671031 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.671035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.671351 (+ 316us) server_negotiation.cc:407] Connection header received
0619 06:55:39.671603 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.671613 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.671717 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.671781 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:39.672536 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.673400 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.675179 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.675961 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.676178 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.676257 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.676506 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.676635 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:55:39.676987 6494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.670547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.66:33329 (local address 127.18.39.65:53225)
0619 06:55:39.671228 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.671254 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.671397 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.672011 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.672015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.672038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:39.672393 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.672400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.673522 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.673530 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.674984 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.674996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.676157 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.676164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.676340 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.676364 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.676470 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.676606 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"spinlock_wait_cycles":57088,"thread_start_us":364,"threads_started":1}
I20260619 06:55:39.677593 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.671654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.39.65:50055 (local address 127.18.39.67:33233)
0619 06:55:39.671906 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.671911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.672131 (+ 220us) server_negotiation.cc:407] Connection header received
0619 06:55:39.672381 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.672386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.672453 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.672553 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:55:39.673146 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.673930 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.675633 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.676618 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.676816 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.676882 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.677210 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.677368 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:55:39.678257 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.671362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.67:33233 (local address 127.18.39.65:50055)
0619 06:55:39.672021 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 06:55:39.672036 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:55:39.672225 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:55:39.672566 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:55:39.672571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:55:39.672593 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:55:39.672992 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.673002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.674080 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.674086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.675441 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:55:39.675452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.676826 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.676832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:55:39.677014 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.677044 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:55:39.677771 (+ 727us) client_negotiation.cc:240] Negotiation successful
0619 06:55:39.677910 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":280,"threads_started":1}
I20260619 06:55:39.678359 6292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fdaed708c449a6999f22ae9ad5ca45" candidate_uuid: "a13b5fc029c74a9ab33a589406acce88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" is_pre_election: true
I20260619 06:55:39.679033 6292 raft_consensus.cc:2454] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a13b5fc029c74a9ab33a589406acce88 in term 0.
I20260619 06:55:39.679203 6422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fdaed708c449a6999f22ae9ad5ca45" candidate_uuid: "a13b5fc029c74a9ab33a589406acce88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d7bb5cdd2243248de14d46777fc64e" is_pre_election: true
I20260619 06:55:39.679716 6422 raft_consensus.cc:2454] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a13b5fc029c74a9ab33a589406acce88 in term 0.
I20260619 06:55:39.679780 6095 leader_election.cc:302] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13b5fc029c74a9ab33a589406acce88, c0613d6b18aa4d4ca2c2e1c56dbd601b; no voters:
I20260619 06:55:39.680214 6493 raft_consensus.cc:2781] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:55:39.680373 6493 raft_consensus.cc:482] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:55:39.680481 6493 raft_consensus.cc:3037] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:39.682775 6493 raft_consensus.cc:504] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.683408 6493 leader_election.cc:288] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [CANDIDATE]: Term 1 election: Requested vote from peers c0613d6b18aa4d4ca2c2e1c56dbd601b (127.18.39.66:33329), 27d7bb5cdd2243248de14d46777fc64e (127.18.39.67:33233)
I20260619 06:55:39.683941 6292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fdaed708c449a6999f22ae9ad5ca45" candidate_uuid: "a13b5fc029c74a9ab33a589406acce88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b"
I20260619 06:55:39.683977 6422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20fdaed708c449a6999f22ae9ad5ca45" candidate_uuid: "a13b5fc029c74a9ab33a589406acce88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d7bb5cdd2243248de14d46777fc64e"
I20260619 06:55:39.684157 6292 raft_consensus.cc:3037] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:39.684209 6422 raft_consensus.cc:3037] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:39.686578 6422 raft_consensus.cc:2454] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a13b5fc029c74a9ab33a589406acce88 in term 1.
I20260619 06:55:39.686580 6292 raft_consensus.cc:2454] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a13b5fc029c74a9ab33a589406acce88 in term 1.
I20260619 06:55:39.687089 6095 leader_election.cc:302] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27d7bb5cdd2243248de14d46777fc64e, a13b5fc029c74a9ab33a589406acce88; no voters:
I20260619 06:55:39.687425 6493 raft_consensus.cc:2781] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:39.687902 6493 raft_consensus.cc:686] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 1 LEADER]: Becoming Leader. State: Replica: a13b5fc029c74a9ab33a589406acce88, State: Running, Role: LEADER
I20260619 06:55:39.688393 6493 consensus_queue.cc:237] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } }
I20260619 06:55:39.693576 6019 catalog_manager.cc:5495] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 reported cstate change: term changed from 0 to 1, leader changed from <none> to a13b5fc029c74a9ab33a589406acce88 (127.18.39.65). New cstate: current_term: 1 leader_uuid: "a13b5fc029c74a9ab33a589406acce88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a13b5fc029c74a9ab33a589406acce88" member_type: VOTER last_known_addr { host: "127.18.39.65" port: 43663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 } health_report { overall_health: UNKNOWN } } }
W20260619 06:55:39.704994 6338 tablet.cc:2369] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:55:39.730365 6207 tablet.cc:2369] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:55:39.735877 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.730700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47824 (local address 127.18.39.65:43663)
0619 06:55:39.730920 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.730928 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.730951 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:55:39.731048 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.731054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.731209 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.731355 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:55:39.731652 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.732423 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.733822 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.734218 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.734310 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.734757 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:55:39.735250 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:55:39.735355 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.735491 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.735595 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:55:39.744367 6422 raft_consensus.cc:1261] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 1 FOLLOWER]: Refusing update from remote peer a13b5fc029c74a9ab33a589406acce88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:39.744362 6292 raft_consensus.cc:1261] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 1 FOLLOWER]: Refusing update from remote peer a13b5fc029c74a9ab33a589406acce88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:55:39.745323 6496 consensus_queue.cc:1035] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0613d6b18aa4d4ca2c2e1c56dbd601b" member_type: VOTER last_known_addr { host: "127.18.39.66" port: 33329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:39.745743 6493 consensus_queue.cc:1035] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27d7bb5cdd2243248de14d46777fc64e" member_type: VOTER last_known_addr { host: "127.18.39.67" port: 33233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:55:39.759462 6499 mvcc.cc:205] Tried to move back new op lower bound from 7298466364382515200 to 7298466364168105984. Current Snapshot: MvccSnapshot[applied={T|T < 7298466364382515200}]
I20260619 06:55:39.763250 6501 mvcc.cc:205] Tried to move back new op lower bound from 7298466364382515200 to 7298466364168105984. Current Snapshot: MvccSnapshot[applied={T|T < 7298466364382515200}]
I20260619 06:55:39.777068 6020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:39.777408 6020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:55:39.781210 6020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f9eb0cfadb942c3ad120aebd071b4eb: Sending DeleteTablet for 3 replicas of tablet 20fdaed708c449a6999f22ae9ad5ca45
I20260619 06:55:39.782497 6141 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:39 UTC) from {username='slave'} at 127.0.0.1:47812
I20260619 06:55:39.782610 6272 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:39 UTC) from {username='slave'} at 127.0.0.1:55046
I20260619 06:55:39.782644 6402 tablet_service.cc:1514] Processing DeleteTablet for tablet 20fdaed708c449a6999f22ae9ad5ca45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:55:39 UTC) from {username='slave'} at 127.0.0.1:57154
I20260619 06:55:39.785084 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6079
I20260619 06:55:39.797976 6509 tablet_replica.cc:330] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: stopping tablet replica
I20260619 06:55:39.798787 6509 raft_consensus.cc:2229] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:55:39.799757 6509 raft_consensus.cc:2258] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:39.802639 6509 ts_tablet_manager.cc:1905] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:39.809908 6510 tablet_replica.cc:330] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: stopping tablet replica
I20260619 06:55:39.810570 6511 tablet_replica.cc:330] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: stopping tablet replica
I20260619 06:55:39.811163 6511 raft_consensus.cc:2229] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:39.811707 6510 raft_consensus.cc:2229] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:55:39.812312 6510 raft_consensus.cc:2258] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:39.812893 6511 raft_consensus.cc:2258] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:55:39.813205 6509 ts_tablet_manager.cc:1918] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:39.813485 6509 log.cc:1196] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-0/wal/wals/20fdaed708c449a6999f22ae9ad5ca45
I20260619 06:55:39.814340 6509 ts_tablet_manager.cc:1939] T 20fdaed708c449a6999f22ae9ad5ca45 P a13b5fc029c74a9ab33a589406acce88: Deleting consensus metadata
I20260619 06:55:39.815829 6511 ts_tablet_manager.cc:1905] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:39.816772 6004 catalog_manager.cc:4841] TS a13b5fc029c74a9ab33a589406acce88 (127.18.39.65:43663): tablet 20fdaed708c449a6999f22ae9ad5ca45 (table test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]) successfully deleted
I20260619 06:55:39.817231 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.786425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.18.39.65:43663)
0619 06:55:39.786625 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.786632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.808188 (+ 21556us) server_negotiation.cc:407] Connection header received
0619 06:55:39.808734 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.808741 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.808853 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.808935 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:39.809696 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.810969 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.812896 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.813564 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.813670 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.815313 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:39.815326 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:39.815343 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:39.816207 (+ 864us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:39.816273 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:39.816300 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:39.816390 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.816676 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.816810 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:55:39.817705 6510 ts_tablet_manager.cc:1905] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:55:39.818578 6201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:39.824736 6510 ts_tablet_manager.cc:1918] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:39.824958 6510 log.cc:1196] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-1/wal/wals/20fdaed708c449a6999f22ae9ad5ca45
I20260619 06:55:39.825538 6510 ts_tablet_manager.cc:1939] T 20fdaed708c449a6999f22ae9ad5ca45 P c0613d6b18aa4d4ca2c2e1c56dbd601b: Deleting consensus metadata
I20260619 06:55:39.826952 6511 ts_tablet_manager.cc:1918] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:55:39.827147 6511 log.cc:1196] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Deleting WAL directory at /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851818886250-18589-0/minicluster-data/ts-2/wal/wals/20fdaed708c449a6999f22ae9ad5ca45
I20260619 06:55:39.827301 6005 catalog_manager.cc:4841] TS c0613d6b18aa4d4ca2c2e1c56dbd601b (127.18.39.66:33329): tablet 20fdaed708c449a6999f22ae9ad5ca45 (table test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]) successfully deleted
I20260619 06:55:39.827754 6511 ts_tablet_manager.cc:1939] T 20fdaed708c449a6999f22ae9ad5ca45 P 27d7bb5cdd2243248de14d46777fc64e: Deleting consensus metadata
I20260619 06:55:39.829737 6005 catalog_manager.cc:4841] TS 27d7bb5cdd2243248de14d46777fc64e (127.18.39.67:33233): tablet 20fdaed708c449a6999f22ae9ad5ca45 (table test-table [id=dec27f3d3ca0483ba87697d52cedc3ae]) successfully deleted
I20260619 06:55:39.926235 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6079
I20260619 06:55:39.945034 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6210
I20260619 06:55:39.951998 6482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:39.945926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.18.39.66:33329)
0619 06:55:39.946104 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:39.946108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:39.946128 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:55:39.946188 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:39.946192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:39.946283 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:39.946374 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:39.946727 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.947454 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.948656 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:39.949054 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:39.949151 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:39.950431 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:39.950440 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:39.950456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:39.951170 (+ 714us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:39.951219 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:39.951239 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:39.951335 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:39.951550 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 06:55:39.951647 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:55:39.952744 6332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:40.044929 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6210
I20260619 06:55:40.062450 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6340
I20260619 06:55:40.069442 6484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:40.063268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.18.39.67:33233)
0619 06:55:40.063446 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:40.063451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:40.063472 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:55:40.063609 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:40.063614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:40.063713 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:40.063812 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:40.064203 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:40.064916 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:40.066276 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:40.066667 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:40.066798 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:40.068033 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:40.068041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:40.068057 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:40.068614 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:40.068660 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:40.068679 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:40.068795 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:40.068890 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 06:55:40.068981 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:55:40.070133 6462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:40.160106 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6340
I20260619 06:55:40.176834 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 5989
I20260619 06:55:40.178037 6050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:40.289713 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 5989
2026-06-19T06:55:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2898 ms)
[----------] 6 tests from EncryptionPolicyTest (21964 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:40 dist-test-slave-1h5s krb5kdc[6519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:40 dist-test-slave-1h5s krb5kdc[6519](info): set up 2 sockets
Jun 19 06:55:40 dist-test-slave-1h5s krb5kdc[6519](info): commencing operation
krb5kdc: starting...
W20260619 06:55:41.418190 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:41 dist-test-slave-1h5s krb5kdc[6519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:41.685783 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:42.026322 6532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:42.026928 6532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:42.027130 6532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:42.034667 6532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:42.034847 6532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:42.034941 6532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:42.035028 6532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:42.035099 6532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:42.045365 6532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:42.046819 6532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:42.048166 6532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:42.060794 6540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:42.060747 6538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:42.060747 6537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:42.061738 6532 server_base.cc:1034] running on GCE node
I20260619 06:55:42.062636 6532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:42.064224 6532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:42.065464 6532 hybrid_clock.cc:648] HybridClock initialized: now 1781852142065425 us; error 59 us; skew 500 ppm
I20260619 06:55:42.065877 6532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:42 dist-test-slave-1h5s krb5kdc[6519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852142, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:42.069198 6532 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:42.070896 6532 webserver.cc:460] Webserver started at http://127.0.0.1:40147/ using document root <none> and password file <none>
I20260619 06:55:42.071657 6532 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:42.071825 6532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:42.072155 6532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:42.075616 6532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "efe813803a674b039719459e0b9f9d99"
format_stamp: "Formatted at 2026-06-19 06:55:42 on dist-test-slave-1h5s"
server_key: "18a727505f847bd2827b6dcdf3381432"
server_key_iv: "98ea3e7d7d6be0a1db97e9628884e7e0"
server_key_version: "encryptionkey@0"
I20260619 06:55:42.076500 6532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "efe813803a674b039719459e0b9f9d99"
format_stamp: "Formatted at 2026-06-19 06:55:42 on dist-test-slave-1h5s"
server_key: "18a727505f847bd2827b6dcdf3381432"
server_key_iv: "98ea3e7d7d6be0a1db97e9628884e7e0"
server_key_version: "encryptionkey@0"
I20260619 06:55:42.082434 6532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260619 06:55:42.086362 6547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:42.087898 6532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:42.088146 6532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "efe813803a674b039719459e0b9f9d99"
format_stamp: "Formatted at 2026-06-19 06:55:42 on dist-test-slave-1h5s"
server_key: "18a727505f847bd2827b6dcdf3381432"
server_key_iv: "98ea3e7d7d6be0a1db97e9628884e7e0"
server_key_version: "encryptionkey@0"
I20260619 06:55:42.088397 6532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:42.117725 6532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:42.122691 6532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:42.123075 6532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:42.144445 6532 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43609
I20260619 06:55:42.144471 6598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43609 every 8 connection(s)
I20260619 06:55:42.146121 6532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:42.150036 6599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:42.152020 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6532
I20260619 06:55:42.152279 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:42.152981 18589 external_mini_cluster.cc:1428] Setting key 328d0d7a75ae51f8a85147e7d9123e18
I20260619 06:55:42.162082 6599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99: Bootstrap starting.
I20260619 06:55:42.165235 6599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:42.166317 6599 log.cc:824] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:55:42 dist-test-slave-1h5s krb5kdc[6519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:55:42.169518 6599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99: No bootstrap required, opened a new log
I20260619 06:55:42.178597 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:42.155380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33221 (local address 127.0.0.1:43609)
0619 06:55:42.156224 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:42.156238 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:42.156284 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:55:42.158103 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:42.158126 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:42.158552 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:42.158973 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:42.160472 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:42.161860 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:42.163210 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:42.163945 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:42.164158 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:42.169065 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:42.169076 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:42.169095 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:42.169144 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:42.173390 (+ 4246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:42.174396 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:42.174406 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:42.174416 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:42.174629 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:42.175088 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:42.175098 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:42.175101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:42.175848 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:42.176183 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:42.176677 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 06:55:42.177059 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":382,"threads_started":1}
I20260619 06:55:42.180706 6599 raft_consensus.cc:348] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } }
I20260619 06:55:42.181084 6599 raft_consensus.cc:374] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:42.181216 6599 raft_consensus.cc:729] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe813803a674b039719459e0b9f9d99, State: Initialized, Role: FOLLOWER
I20260619 06:55:42.181903 6599 consensus_queue.cc:260] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } }
I20260619 06:55:42.182228 6599 raft_consensus.cc:388] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:42.182372 6599 raft_consensus.cc:482] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:42.182585 6599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:42.183483 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:42.185854 6599 raft_consensus.cc:504] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } }
I20260619 06:55:42.186553 6599 leader_election.cc:302] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efe813803a674b039719459e0b9f9d99; no voters:
I20260619 06:55:42.187261 6599 leader_election.cc:288] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:42.187399 6604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:42.190019 6604 raft_consensus.cc:686] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [term 1 LEADER]: Becoming Leader. State: Replica: efe813803a674b039719459e0b9f9d99, State: Running, Role: LEADER
I20260619 06:55:42.191071 6599 sys_catalog.cc:564] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:42.190874 6604 consensus_queue.cc:237] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } }
I20260619 06:55:42.200573 6607 sys_catalog.cc:455] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efe813803a674b039719459e0b9f9d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } } }
I20260619 06:55:42.201120 6607 sys_catalog.cc:458] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:42.201998 6609 sys_catalog.cc:455] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efe813803a674b039719459e0b9f9d99. Latest consensus state: current_term: 1 leader_uuid: "efe813803a674b039719459e0b9f9d99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efe813803a674b039719459e0b9f9d99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43609 } } }
I20260619 06:55:42.202329 6622 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:42.202406 6609 sys_catalog.cc:458] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:42.209095 6622 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:42.222597 6602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:42.191249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46389 (local address 127.0.0.1:43609)
0619 06:55:42.191476 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:42.191486 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:42.198861 (+ 7375us) server_negotiation.cc:407] Connection header received
0619 06:55:42.199222 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:42.199230 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:42.199345 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:42.199445 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:42.201403 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:42.202727 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:42.204702 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:42.205899 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:42.206123 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:42.211062 (+ 4939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:42.211074 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:42.211083 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:42.211167 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:42.216910 (+ 5743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:42.218461 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:42.218471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:42.218477 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:42.218681 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:42.219386 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:42.219394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:42.219400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:42.219988 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:42.220313 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:42.221793 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0619 06:55:42.222060 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:55:42.227782 6622 catalog_manager.cc:1347] Generated new cluster ID: 634b6d87510948e79c63e2a57b1dd330
I20260619 06:55:42.227893 6622 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:42.254917 6622 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:42.256320 6622 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:42.262485 6622 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efe813803a674b039719459e0b9f9d99: Generated new TSK 0
I20260619 06:55:42.263406 6622 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:42.277984 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6532
I20260619 06:55:42.279414 6594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:42.387343 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6532
W20260619 06:55:42.406684 6517 connection.cc:535] client connection to 127.0.0.1:43609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43609): Connection reset by peer (error 104)
2026-06-19T06:55:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2094 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:42 dist-test-slave-1h5s krb5kdc[6631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:42 dist-test-slave-1h5s krb5kdc[6631](info): set up 2 sockets
Jun 19 06:55:42 dist-test-slave-1h5s krb5kdc[6631](info): commencing operation
krb5kdc: starting...
W20260619 06:55:43.493808 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:43 dist-test-slave-1h5s krb5kdc[6631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:43.756650 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36287
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36287
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:44.098855 6644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:44.099234 6644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:44.099393 6644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:44.107007 6644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:44.107173 6644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:44.107285 6644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:44.107396 6644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:44.107474 6644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:44.117897 6644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36287
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:44.119292 6644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:44.120602 6644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:44.133010 6649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:44.132942 6650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:44.132977 6652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:44.133780 6644 server_base.cc:1034] running on GCE node
I20260619 06:55:44.134605 6644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:44.136092 6644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:44.137395 6644 hybrid_clock.cc:648] HybridClock initialized: now 1781852144137289 us; error 48 us; skew 500 ppm
I20260619 06:55:44.137768 6644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:44 dist-test-slave-1h5s krb5kdc[6631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:44.145991 6644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:44.147688 6644 webserver.cc:460] Webserver started at http://127.0.0.1:36077/ using document root <none> and password file <none>
I20260619 06:55:44.148428 6644 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:44.148579 6644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:44.148893 6644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:44.152125 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "fd36f232d4624b6489326566cdaa8e33"
format_stamp: "Formatted at 2026-06-19 06:55:44 on dist-test-slave-1h5s"
server_key: "024501aaac81b66650cfadc35f5d1ab5"
server_key_iv: "04c9b9b465e196cd86a53497f4b1307b"
server_key_version: "encryptionkey@0"
I20260619 06:55:44.152990 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "fd36f232d4624b6489326566cdaa8e33"
format_stamp: "Formatted at 2026-06-19 06:55:44 on dist-test-slave-1h5s"
server_key: "024501aaac81b66650cfadc35f5d1ab5"
server_key_iv: "04c9b9b465e196cd86a53497f4b1307b"
server_key_version: "encryptionkey@0"
I20260619 06:55:44.158560 6644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:55:44.162343 6659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:44.163828 6644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:44.164034 6644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "fd36f232d4624b6489326566cdaa8e33"
format_stamp: "Formatted at 2026-06-19 06:55:44 on dist-test-slave-1h5s"
server_key: "024501aaac81b66650cfadc35f5d1ab5"
server_key_iv: "04c9b9b465e196cd86a53497f4b1307b"
server_key_version: "encryptionkey@0"
I20260619 06:55:44.164270 6644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:44.203457 6644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:44.208209 6644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:44.208537 6644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:44.228148 6644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36287
I20260619 06:55:44.228181 6710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36287 every 8 connection(s)
I20260619 06:55:44.229722 6644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:44.233522 6711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:44.234246 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6644
I20260619 06:55:44.234438 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:44.235018 18589 external_mini_cluster.cc:1428] Setting key 286f2b8086ab9c4c7ae587e97577309f
I20260619 06:55:44.248030 6711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33: Bootstrap starting.
I20260619 06:55:44.249622 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:44.236843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39007 (local address 127.0.0.1:36287)
0619 06:55:44.237711 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:44.237722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:44.237763 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:55:44.239914 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:44.239943 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:44.240387 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:44.240772 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:44.241410 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:44.242700 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:44.244623 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:44.245400 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:44.245553 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:44.246972 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:44.246980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:44.246995 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:44.247539 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:44.247617 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:44.247641 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:44.247769 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:44.248233 (+ 464us) server_negotiation.cc:299] Negotiation successful
0619 06:55:44.248449 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":421,"threads_started":1}
I20260619 06:55:44.251525 6711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:44.252761 6711 log.cc:824] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:44.254098 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:44.256000 6711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33: No bootstrap required, opened a new log
I20260619 06:55:44.265252 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:44.258731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56697 (local address 127.0.0.1:36287)
0619 06:55:44.258931 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:44.258940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:44.259339 (+ 399us) server_negotiation.cc:407] Connection header received
0619 06:55:44.259574 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:44.259577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:44.259645 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:44.259761 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:44.260284 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:44.261081 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:44.262487 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:44.263008 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:44.263161 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:44.264405 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:44.264410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:44.264413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:44.264433 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:44.264488 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:44.264504 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:44.264638 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:44.264885 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 06:55:44.264942 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:55:44.265195 6711 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } }
I20260619 06:55:44.265849 6711 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:44.265972 6711 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd36f232d4624b6489326566cdaa8e33, State: Initialized, Role: FOLLOWER
I20260619 06:55:44.266705 6711 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } }
I20260619 06:55:44.267019 6711 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:44.267174 6711 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:44.267347 6711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:44.269853 6711 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } }
I20260619 06:55:44.270414 6711 leader_election.cc:302] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd36f232d4624b6489326566cdaa8e33; no voters:
I20260619 06:55:44.271003 6711 leader_election.cc:288] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:44.271193 6721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:44.272583 6721 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [term 1 LEADER]: Becoming Leader. State: Replica: fd36f232d4624b6489326566cdaa8e33, State: Running, Role: LEADER
I20260619 06:55:44.273144 6721 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } }
I20260619 06:55:44.273277 6711 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:44.276036 6723 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd36f232d4624b6489326566cdaa8e33. Latest consensus state: current_term: 1 leader_uuid: "fd36f232d4624b6489326566cdaa8e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } } }
I20260619 06:55:44.276389 6723 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:44.277393 6722 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd36f232d4624b6489326566cdaa8e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd36f232d4624b6489326566cdaa8e33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36287 } } }
I20260619 06:55:44.277762 6722 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:44.288385 6729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:44.293852 6729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:44.303378 6729 catalog_manager.cc:1347] Generated new cluster ID: a2266e0a153c4e0787d7b0715e1e9c46
I20260619 06:55:44.303512 6729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:44.313879 6729 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:44.315413 6729 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:44.327998 6729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd36f232d4624b6489326566cdaa8e33: Generated new TSK 0
I20260619 06:55:44.328976 6729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:44.343918 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6644
W20260619 06:55:44.345165 6706 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39007
W20260619 06:55:44.345986 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6644: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:44.346164 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6644
W20260619 06:55:44.365744 6629 connection.cc:535] client connection to 127.0.0.1:36287 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36287): Connection reset by peer (error 104)
2026-06-19T06:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1958 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:44 dist-test-slave-1h5s krb5kdc[6742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:44 dist-test-slave-1h5s krb5kdc[6742](info): set up 2 sockets
Jun 19 06:55:44 dist-test-slave-1h5s krb5kdc[6742](info): commencing operation
krb5kdc: starting...
W20260619 06:55:45.449793 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:45 dist-test-slave-1h5s krb5kdc[6742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:45.710389 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:46.052187 6755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:46.052515 6755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:46.052611 6755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:46.059969 6755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:46.060105 6755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:46.060153 6755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:46.060220 6755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:46.060266 6755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:46.070423 6755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39837
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:46.071755 6755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:46.073055 6755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:46.085354 6760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:46.085307 6761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:46.085276 6763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:46.085973 6755 server_base.cc:1034] running on GCE node
I20260619 06:55:46.087006 6755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:46.089432 6755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:46.090708 6755 hybrid_clock.cc:648] HybridClock initialized: now 1781852146090627 us; error 52 us; skew 500 ppm
I20260619 06:55:46.091216 6755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:46 dist-test-slave-1h5s krb5kdc[6742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:46.096138 6755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:46.098516 6755 webserver.cc:460] Webserver started at http://127.0.0.1:32855/ using document root <none> and password file <none>
I20260619 06:55:46.099771 6755 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:46.099920 6755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:46.100579 6755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:46.105504 6755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "18fc223f5fea407c9441da1bb54b3c61"
format_stamp: "Formatted at 2026-06-19 06:55:46 on dist-test-slave-1h5s"
server_key: "e4efb0fdd3fd93e022fe1f6f2bf5993c"
server_key_iv: "b90e5e08fca3a742eea554d9f0df7cc5"
server_key_version: "encryptionkey@0"
I20260619 06:55:46.106639 6755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "18fc223f5fea407c9441da1bb54b3c61"
format_stamp: "Formatted at 2026-06-19 06:55:46 on dist-test-slave-1h5s"
server_key: "e4efb0fdd3fd93e022fe1f6f2bf5993c"
server_key_iv: "b90e5e08fca3a742eea554d9f0df7cc5"
server_key_version: "encryptionkey@0"
I20260619 06:55:46.114300 6755 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.003s
I20260619 06:55:46.119439 6770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:46.121564 6755 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 06:55:46.121767 6755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "18fc223f5fea407c9441da1bb54b3c61"
format_stamp: "Formatted at 2026-06-19 06:55:46 on dist-test-slave-1h5s"
server_key: "e4efb0fdd3fd93e022fe1f6f2bf5993c"
server_key_iv: "b90e5e08fca3a742eea554d9f0df7cc5"
server_key_version: "encryptionkey@0"
I20260619 06:55:46.122020 6755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:46.130162 6755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:46.136123 6755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:46.136487 6755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:46.156054 6755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39837
I20260619 06:55:46.156095 6821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39837 every 8 connection(s)
I20260619 06:55:46.157567 6755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:46.161290 6822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:46.166118 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6755
I20260619 06:55:46.166316 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:46.166769 18589 external_mini_cluster.cc:1428] Setting key cec59ad7f9d7b9ca08d4354501dfb316
I20260619 06:55:46.174060 6822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61: Bootstrap starting.
I20260619 06:55:46.176195 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:46.168637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55007 (local address 127.0.0.1:39837)
0619 06:55:46.169500 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:46.169520 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:46.169573 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:55:46.172108 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:46.172140 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:46.172620 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:46.173098 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:46.174181 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:46.174189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:46.174204 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:46.174222 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:46.174294 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:46.174335 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:46.174563 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:46.174711 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 06:55:46.174905 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":361,"threads_started":1}
I20260619 06:55:46.177971 6822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:46.179304 6822 log.cc:824] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:46.180466 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:46.182837 6822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61: No bootstrap required, opened a new log
I20260619 06:55:46.188905 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:46.185943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36025 (local address 127.0.0.1:39837)
0619 06:55:46.186185 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:46.186201 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:46.186547 (+ 346us) server_negotiation.cc:407] Connection header received
0619 06:55:46.186747 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:46.186751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:46.186807 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:46.186894 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:46.188130 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:46.188136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:46.188142 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:46.188165 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:46.188232 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:46.188251 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:46.188349 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:46.188498 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 06:55:46.188554 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:55:46.192754 6822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } }
I20260619 06:55:46.193156 6822 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:46.193233 6822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18fc223f5fea407c9441da1bb54b3c61, State: Initialized, Role: FOLLOWER
I20260619 06:55:46.193945 6822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } }
I20260619 06:55:46.194180 6822 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:46.194279 6822 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:46.194406 6822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:46.196713 6822 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } }
I20260619 06:55:46.197207 6822 leader_election.cc:302] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18fc223f5fea407c9441da1bb54b3c61; no voters:
I20260619 06:55:46.197875 6822 leader_election.cc:288] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:46.198055 6832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:46.200003 6822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:46.200459 6832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [term 1 LEADER]: Becoming Leader. State: Replica: 18fc223f5fea407c9441da1bb54b3c61, State: Running, Role: LEADER
I20260619 06:55:46.201011 6832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } }
I20260619 06:55:46.204210 6834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18fc223f5fea407c9441da1bb54b3c61. Latest consensus state: current_term: 1 leader_uuid: "18fc223f5fea407c9441da1bb54b3c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } } }
I20260619 06:55:46.204622 6834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:46.204499 6833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18fc223f5fea407c9441da1bb54b3c61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18fc223f5fea407c9441da1bb54b3c61" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39837 } } }
I20260619 06:55:46.204887 6833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:46.207506 6844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:46.211887 6844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:46.220422 6844 catalog_manager.cc:1347] Generated new cluster ID: 97bfcb34b0bc461cbe849ae00a56b029
I20260619 06:55:46.220556 6844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:46.240159 6844 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:46.240715 6844 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:46.257730 6844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18fc223f5fea407c9441da1bb54b3c61: Generated new TSK 0
I20260619 06:55:46.258867 6844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:46.270732 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6755
W20260619 06:55:46.271898 6817 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55007
W20260619 06:55:46.272679 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6755: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:46.272836 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6755
2026-06-19T06:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1923 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:46 dist-test-slave-1h5s krb5kdc[6853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:46 dist-test-slave-1h5s krb5kdc[6853](info): set up 2 sockets
Jun 19 06:55:46 dist-test-slave-1h5s krb5kdc[6853](info): commencing operation
krb5kdc: starting...
W20260619 06:55:47.397733 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:47 dist-test-slave-1h5s krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:47.661633 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:47.996678 6866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:47.996997 6866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:47.997092 6866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:48.004477 6866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:48.004606 6866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:48.004652 6866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:48.004706 6866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:48.004750 6866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:48.015740 6866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:48.017022 6866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:48.018313 6866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:48.030433 6871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:48.030340 6874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:48.030380 6872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:48.031206 6866 server_base.cc:1034] running on GCE node
I20260619 06:55:48.032018 6866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:48.033578 6866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:48.034799 6866 hybrid_clock.cc:648] HybridClock initialized: now 1781852148034768 us; error 34 us; skew 500 ppm
I20260619 06:55:48.035143 6866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:48 dist-test-slave-1h5s krb5kdc[6853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:48.038266 6866 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:48.039677 6866 webserver.cc:460] Webserver started at http://127.0.0.1:32847/ using document root <none> and password file <none>
I20260619 06:55:48.040337 6866 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:48.040484 6866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:48.040786 6866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:48.043776 6866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "6fad57e30ba5446c9b89a2e45419938d"
format_stamp: "Formatted at 2026-06-19 06:55:48 on dist-test-slave-1h5s"
server_key: "e4005a34a2fa56df110e0fa9ef75bb5d"
server_key_iv: "b72cbe6f1ad49e331ef4b4e3f8608d36"
server_key_version: "encryptionkey@0"
I20260619 06:55:48.044567 6866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "6fad57e30ba5446c9b89a2e45419938d"
format_stamp: "Formatted at 2026-06-19 06:55:48 on dist-test-slave-1h5s"
server_key: "e4005a34a2fa56df110e0fa9ef75bb5d"
server_key_iv: "b72cbe6f1ad49e331ef4b4e3f8608d36"
server_key_version: "encryptionkey@0"
I20260619 06:55:48.050194 6866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 06:55:48.054196 6881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:48.055610 6866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:48.055840 6866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "6fad57e30ba5446c9b89a2e45419938d"
format_stamp: "Formatted at 2026-06-19 06:55:48 on dist-test-slave-1h5s"
server_key: "e4005a34a2fa56df110e0fa9ef75bb5d"
server_key_iv: "b72cbe6f1ad49e331ef4b4e3f8608d36"
server_key_version: "encryptionkey@0"
I20260619 06:55:48.056061 6866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:48.076418 6866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:48.080592 6866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:48.080909 6866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:48.100101 6866 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41921
I20260619 06:55:48.100136 6932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41921 every 8 connection(s)
I20260619 06:55:48.101632 6866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:48.105258 6933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:48.107734 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6866
I20260619 06:55:48.107918 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:48.108466 18589 external_mini_cluster.cc:1428] Setting key ce2a701e88d07cf53b242583c55f9177
I20260619 06:55:48.117064 6933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d: Bootstrap starting.
I20260619 06:55:48.120194 6933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:48.121189 6933 log.cc:824] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:55:48 dist-test-slave-1h5s krb5kdc[6853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:55:48.124197 6933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d: No bootstrap required, opened a new log
I20260619 06:55:48.130772 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:48.110408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50701 (local address 127.0.0.1:41921)
0619 06:55:48.111169 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:48.111180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:48.111224 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:55:48.112906 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:48.112927 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:48.113306 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:48.113751 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:48.115231 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:48.116515 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:48.117938 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:48.118635 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:48.118857 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:48.123328 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:48.123336 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:48.123351 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:48.123393 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:48.126590 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:48.127352 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:48.127359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:48.127368 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:48.127547 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:48.127982 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:48.127991 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:48.127994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:48.128539 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:48.128903 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:48.129327 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 06:55:48.129766 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":325,"threads_started":1}
I20260619 06:55:48.132501 6933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 06:55:48.132951 6933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:48.133055 6933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fad57e30ba5446c9b89a2e45419938d, State: Initialized, Role: FOLLOWER
I20260619 06:55:48.133701 6933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 06:55:48.133986 6933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:48.134127 6933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:48.134301 6933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:48.134976 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:48.136667 6933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 06:55:48.137174 6933 leader_election.cc:302] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fad57e30ba5446c9b89a2e45419938d; no voters:
I20260619 06:55:48.137897 6933 leader_election.cc:288] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:48.138000 6939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:48.140141 6939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [term 1 LEADER]: Becoming Leader. State: Replica: 6fad57e30ba5446c9b89a2e45419938d, State: Running, Role: LEADER
I20260619 06:55:48.140146 6933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:48.140779 6939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } }
I20260619 06:55:48.144040 6941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fad57e30ba5446c9b89a2e45419938d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } } }
I20260619 06:55:48.144510 6941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:48.145116 6942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fad57e30ba5446c9b89a2e45419938d. Latest consensus state: current_term: 1 leader_uuid: "6fad57e30ba5446c9b89a2e45419938d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fad57e30ba5446c9b89a2e45419938d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41921 } } }
I20260619 06:55:48.145665 6942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:48.147851 6949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:48.153074 6949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:48.158586 6936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:48.140868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36723 (local address 127.0.0.1:41921)
0619 06:55:48.141053 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:48.141062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:48.141080 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:55:48.141409 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:48.141414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:48.141488 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:48.141606 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:48.143570 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:48.144678 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:48.146737 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:48.147502 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:48.147622 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:48.150938 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:48.150945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:48.150949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:48.150987 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:48.155027 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:48.155961 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:48.155967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:48.155971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:48.156125 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:48.156620 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:48.156626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:48.156630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:48.157094 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:48.157358 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:48.157955 (+ 597us) server_negotiation.cc:299] Negotiation successful
0619 06:55:48.158198 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:55:48.166901 6949 catalog_manager.cc:1347] Generated new cluster ID: eb66508d8d234685843c85319d338bf1
I20260619 06:55:48.167016 6949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:48.197669 6949 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:48.199030 6949 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:48.204975 6949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fad57e30ba5446c9b89a2e45419938d: Generated new TSK 0
I20260619 06:55:48.206072 6949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:48.220782 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6866
I20260619 06:55:48.222065 6928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:48.317795 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6866
2026-06-19T06:55:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:48 dist-test-slave-1h5s krb5kdc[6965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:48 dist-test-slave-1h5s krb5kdc[6965](info): set up 2 sockets
Jun 19 06:55:48 dist-test-slave-1h5s krb5kdc[6965](info): commencing operation
krb5kdc: starting...
W20260619 06:55:49.421842 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:49 dist-test-slave-1h5s krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:49.685678 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44483
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:50.030121 6978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:50.030480 6978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:50.030658 6978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:50.038038 6978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:50.038218 6978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:50.038339 6978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:50.038458 6978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:50.038548 6978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:50.048663 6978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:50.050139 6978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:50.051483 6978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:50.063179 6984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:50.063289 6983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:50.063537 6986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:50.064261 6978 server_base.cc:1034] running on GCE node
I20260619 06:55:50.065076 6978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:50.066490 6978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:50.067760 6978 hybrid_clock.cc:648] HybridClock initialized: now 1781852150067715 us; error 54 us; skew 500 ppm
I20260619 06:55:50.068116 6978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:50 dist-test-slave-1h5s krb5kdc[6965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:50.074616 6978 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:50.076236 6978 webserver.cc:460] Webserver started at http://127.0.0.1:42211/ using document root <none> and password file <none>
I20260619 06:55:50.076936 6978 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:50.077091 6978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:50.077481 6978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:50.080565 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "b6a9ea86e8004f53a7e0e23e14c721f4"
format_stamp: "Formatted at 2026-06-19 06:55:50 on dist-test-slave-1h5s"
server_key: "f476886ad36498756a9f4733fa9617fc"
server_key_iv: "f666e41a28df3d0c8698a47040df91a0"
server_key_version: "encryptionkey@0"
I20260619 06:55:50.081426 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "b6a9ea86e8004f53a7e0e23e14c721f4"
format_stamp: "Formatted at 2026-06-19 06:55:50 on dist-test-slave-1h5s"
server_key: "f476886ad36498756a9f4733fa9617fc"
server_key_iv: "f666e41a28df3d0c8698a47040df91a0"
server_key_version: "encryptionkey@0"
I20260619 06:55:50.086602 6978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260619 06:55:50.090060 6993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:50.091513 6978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:55:50.091732 6978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "b6a9ea86e8004f53a7e0e23e14c721f4"
format_stamp: "Formatted at 2026-06-19 06:55:50 on dist-test-slave-1h5s"
server_key: "f476886ad36498756a9f4733fa9617fc"
server_key_iv: "f666e41a28df3d0c8698a47040df91a0"
server_key_version: "encryptionkey@0"
I20260619 06:55:50.091953 6978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:50.106901 6978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:50.111521 6978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:50.111852 6978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:50.131099 6978 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44483
I20260619 06:55:50.131122 7044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44483 every 8 connection(s)
I20260619 06:55:50.132536 6978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:50.136619 7045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:50.142469 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 6978
I20260619 06:55:50.142661 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:50.143188 18589 external_mini_cluster.cc:1428] Setting key de5ca240f94eb25f40b56d19d0bc3dd6
I20260619 06:55:50.148730 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4: Bootstrap starting.
I20260619 06:55:50.152599 7045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:50.153956 7045 log.cc:824] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:50.157231 7045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4: No bootstrap required, opened a new log
I20260619 06:55:50.157531 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:50.145086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43019 (local address 127.0.0.1:44483)
0619 06:55:50.146020 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:50.146035 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:50.146080 (+ 45us) server_negotiation.cc:407] Connection header received
0619 06:55:50.147858 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:50.147883 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:50.148295 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:50.148671 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:50.149302 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:50.150650 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:50.152569 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:50.153475 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:50.153612 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:50.154894 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:50.154906 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:50.154932 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:50.155658 (+ 726us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:50.155798 (+ 140us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:50.155826 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:50.155906 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:50.156185 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 06:55:50.156498 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":472,"threads_started":1}
I20260619 06:55:50.161309 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:50.167348 7045 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } }
I20260619 06:55:50.167869 7045 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:50.167980 7045 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a9ea86e8004f53a7e0e23e14c721f4, State: Initialized, Role: FOLLOWER
I20260619 06:55:50.168658 7045 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } }
I20260619 06:55:50.168929 7045 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:50.169093 7045 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:50.169292 7045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:50.171607 7045 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } }
I20260619 06:55:50.172148 7045 leader_election.cc:302] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6a9ea86e8004f53a7e0e23e14c721f4; no voters:
I20260619 06:55:50.172943 7045 leader_election.cc:288] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:50.173053 7055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:50.173637 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:50.166894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40203 (local address 127.0.0.1:44483)
0619 06:55:50.167056 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:50.167060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:50.167417 (+ 357us) server_negotiation.cc:407] Connection header received
0619 06:55:50.167664 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:50.167668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:50.167745 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:50.167901 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:50.168567 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:50.169592 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:50.170910 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:50.171374 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:50.171509 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:50.172746 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:50.172751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:50.172754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:50.172774 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:50.172818 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:50.172834 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:50.172952 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:50.173107 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 06:55:50.173191 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 06:55:50.173804 7055 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [term 1 LEADER]: Becoming Leader. State: Replica: b6a9ea86e8004f53a7e0e23e14c721f4, State: Running, Role: LEADER
I20260619 06:55:50.174629 7055 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } }
I20260619 06:55:50.177102 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6a9ea86e8004f53a7e0e23e14c721f4. Latest consensus state: current_term: 1 leader_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } } }
I20260619 06:55:50.177559 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:50.177943 7057 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6a9ea86e8004f53a7e0e23e14c721f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44483 } } }
I20260619 06:55:50.178332 7057 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:50.179119 7045 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:50.179800 7059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:50.186916 7059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:50.196552 7059 catalog_manager.cc:1347] Generated new cluster ID: 62f96f27f2fb47cb88b6c5e19e976e3b
I20260619 06:55:50.196688 7059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:50.219517 7059 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:50.220952 7059 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:50.229076 7059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6a9ea86e8004f53a7e0e23e14c721f4: Generated new TSK 0
I20260619 06:55:50.230055 7059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:50.239149 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6978
W20260619 06:55:50.240360 7040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43019
W20260619 06:55:50.241137 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 6978: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:50.241323 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 6978
2026-06-19T06:55:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1922 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:50 dist-test-slave-1h5s krb5kdc[7076](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:50 dist-test-slave-1h5s krb5kdc[7076](info): set up 2 sockets
Jun 19 06:55:50 dist-test-slave-1h5s krb5kdc[7076](info): commencing operation
krb5kdc: starting...
W20260619 06:55:51.353937 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:51 dist-test-slave-1h5s krb5kdc[7076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:51.618536 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:51.956995 7089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:51.957459 7089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:51.957633 7089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:51.965322 7089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:51.965525 7089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:51.965621 7089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:51.965734 7089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:51.965801 7089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:51.976375 7089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:51.977885 7089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:51.979269 7089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:51.991848 7097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:51.991930 7095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:51.991873 7094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:51.992944 7089 server_base.cc:1034] running on GCE node
I20260619 06:55:51.993849 7089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:51.995433 7089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:51.996757 7089 hybrid_clock.cc:648] HybridClock initialized: now 1781852151996652 us; error 75 us; skew 500 ppm
I20260619 06:55:51.997207 7089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:51 dist-test-slave-1h5s krb5kdc[7076](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:52.000682 7089 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:52.002424 7089 webserver.cc:460] Webserver started at http://127.0.0.1:41009/ using document root <none> and password file <none>
I20260619 06:55:52.003238 7089 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:52.003391 7089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:52.003716 7089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:52.006937 7089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "28d2bb663a0c48db864a5ec2e87f6679"
format_stamp: "Formatted at 2026-06-19 06:55:51 on dist-test-slave-1h5s"
server_key: "c93aac140bd561cbda4620ae2ab0a19b"
server_key_iv: "47e9f77974f9c0d35bcef757a6395a3e"
server_key_version: "encryptionkey@0"
I20260619 06:55:52.007774 7089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "28d2bb663a0c48db864a5ec2e87f6679"
format_stamp: "Formatted at 2026-06-19 06:55:51 on dist-test-slave-1h5s"
server_key: "c93aac140bd561cbda4620ae2ab0a19b"
server_key_iv: "47e9f77974f9c0d35bcef757a6395a3e"
server_key_version: "encryptionkey@0"
I20260619 06:55:52.013253 7089 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:55:52.016983 7104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:52.018543 7089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:55:52.018752 7089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "28d2bb663a0c48db864a5ec2e87f6679"
format_stamp: "Formatted at 2026-06-19 06:55:51 on dist-test-slave-1h5s"
server_key: "c93aac140bd561cbda4620ae2ab0a19b"
server_key_iv: "47e9f77974f9c0d35bcef757a6395a3e"
server_key_version: "encryptionkey@0"
I20260619 06:55:52.018985 7089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:52.025134 7089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:52.029614 7089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:52.029950 7089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:52.049186 7089 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45775
I20260619 06:55:52.049227 7155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45775 every 8 connection(s)
I20260619 06:55:52.050776 7089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:52.054356 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7089
I20260619 06:55:52.054553 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:52.055011 18589 external_mini_cluster.cc:1428] Setting key e310863e21ff4be1f06c0a84009a8bb1
I20260619 06:55:52.055073 7156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:52.063920 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:52.057026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36625 (local address 127.0.0.1:45775)
0619 06:55:52.058083 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:52.058100 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:52.058690 (+ 590us) server_negotiation.cc:407] Connection header received
0619 06:55:52.060280 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:52.060301 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:52.060648 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:52.061008 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:52.062224 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:52.062231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:52.062245 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:52.062261 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:52.062330 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:52.062351 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:52.062540 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:52.062652 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 06:55:52.062914 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":473,"threads_started":1}
I20260619 06:55:52.067867 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:52.069080 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679: Bootstrap starting.
I20260619 06:55:52.072456 7156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:52.073860 7156 log.cc:824] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:52.075273 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:52.072582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55039 (local address 127.0.0.1:45775)
0619 06:55:52.072837 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:52.072845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:52.073150 (+ 305us) server_negotiation.cc:407] Connection header received
0619 06:55:52.073405 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:52.073414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:52.073467 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:52.073580 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:52.074563 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:52.074572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:52.074577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:52.074595 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:52.074649 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:52.074665 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:52.074803 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:52.074894 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 06:55:52.074959 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:55:52.077055 7156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679: No bootstrap required, opened a new log
I20260619 06:55:52.084849 7156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260619 06:55:52.085309 7156 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:52.085480 7156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d2bb663a0c48db864a5ec2e87f6679, State: Initialized, Role: FOLLOWER
I20260619 06:55:52.086089 7156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260619 06:55:52.086369 7156 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:52.086530 7156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:52.086706 7156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:52.089164 7156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260619 06:55:52.089780 7156 leader_election.cc:302] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d2bb663a0c48db864a5ec2e87f6679; no voters:
I20260619 06:55:52.090371 7156 leader_election.cc:288] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:52.090541 7166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:52.091219 7166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [term 1 LEADER]: Becoming Leader. State: Replica: 28d2bb663a0c48db864a5ec2e87f6679, State: Running, Role: LEADER
I20260619 06:55:52.091809 7166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260619 06:55:52.092584 7156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:52.094947 7167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d2bb663a0c48db864a5ec2e87f6679" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } } }
I20260619 06:55:52.094920 7168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d2bb663a0c48db864a5ec2e87f6679. Latest consensus state: current_term: 1 leader_uuid: "28d2bb663a0c48db864a5ec2e87f6679" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d2bb663a0c48db864a5ec2e87f6679" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } } }
I20260619 06:55:52.095479 7167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:52.095489 7168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:52.099318 7173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:52.104700 7173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:52.115046 7173 catalog_manager.cc:1347] Generated new cluster ID: afffe57e876249ea9aac77385a81abd4
I20260619 06:55:52.115197 7173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:52.130816 7173 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:52.131322 7173 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:52.139595 7173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d2bb663a0c48db864a5ec2e87f6679: Generated new TSK 0
I20260619 06:55:52.140739 7173 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:52.158911 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7089
W20260619 06:55:52.160058 7151 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36625
W20260619 06:55:52.160806 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7089: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:52.160964 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7089
2026-06-19T06:55:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1919 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:52 dist-test-slave-1h5s krb5kdc[7187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:52 dist-test-slave-1h5s krb5kdc[7187](info): set up 2 sockets
Jun 19 06:55:52 dist-test-slave-1h5s krb5kdc[7187](info): commencing operation
krb5kdc: starting...
W20260619 06:55:53.269865 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:53 dist-test-slave-1h5s krb5kdc[7187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:53.537400 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:53.880720 7200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:53.881042 7200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:53.881230 7200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:53.889338 7200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:53.889611 7200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:53.889729 7200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:53.889837 7200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:53.889911 7200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:53.903319 7200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:53.904726 7200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:53.906144 7200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:53.918828 7206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:53.918908 7205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:53.918931 7208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:53.919770 7200 server_base.cc:1034] running on GCE node
I20260619 06:55:53.920616 7200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:53.922303 7200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:53.923516 7200 hybrid_clock.cc:648] HybridClock initialized: now 1781852153923484 us; error 31 us; skew 500 ppm
I20260619 06:55:53.923930 7200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:53 dist-test-slave-1h5s krb5kdc[7187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852153, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:53.927320 7200 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:53.928977 7200 webserver.cc:460] Webserver started at http://127.0.0.1:37181/ using document root <none> and password file <none>
I20260619 06:55:53.929812 7200 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:53.929951 7200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:53.930305 7200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:53.933683 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "2c1256b5d3f24352a8e25cf15614ecd6"
format_stamp: "Formatted at 2026-06-19 06:55:53 on dist-test-slave-1h5s"
server_key: "bf5e00cc434e12b4eb18502e2cacd9c3"
server_key_iv: "2625e5ff345c7884171e32a4f42f2c51"
server_key_version: "encryptionkey@0"
I20260619 06:55:53.934526 7200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "2c1256b5d3f24352a8e25cf15614ecd6"
format_stamp: "Formatted at 2026-06-19 06:55:53 on dist-test-slave-1h5s"
server_key: "bf5e00cc434e12b4eb18502e2cacd9c3"
server_key_iv: "2625e5ff345c7884171e32a4f42f2c51"
server_key_version: "encryptionkey@0"
I20260619 06:55:53.940037 7200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260619 06:55:53.943840 7215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:53.945417 7200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:55:53.945619 7200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "2c1256b5d3f24352a8e25cf15614ecd6"
format_stamp: "Formatted at 2026-06-19 06:55:53 on dist-test-slave-1h5s"
server_key: "bf5e00cc434e12b4eb18502e2cacd9c3"
server_key_iv: "2625e5ff345c7884171e32a4f42f2c51"
server_key_version: "encryptionkey@0"
I20260619 06:55:53.945843 7200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:53.966012 7200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:53.970528 7200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:53.970856 7200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:53.990360 7200 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41007
I20260619 06:55:53.990388 7266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41007 every 8 connection(s)
I20260619 06:55:53.991887 7200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:53.993701 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7200
I20260619 06:55:53.993939 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:53.994372 18589 external_mini_cluster.cc:1428] Setting key 95742ae66964389ec1327a040686f3e9
I20260619 06:55:53.996273 7267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:55:54 dist-test-slave-1h5s krb5kdc[7187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:55:54.010814 7267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6: Bootstrap starting.
I20260619 06:55:54.015012 7267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:54.016430 7267 log.cc:824] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:54.019639 7267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6: No bootstrap required, opened a new log
I20260619 06:55:54.021039 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:53.996404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:53405 (local address 127.0.0.1:41007)
0619 06:55:53.997228 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:53.997246 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:53.997300 (+ 54us) server_negotiation.cc:407] Connection header received
0619 06:55:53.999599 (+ 2299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:53.999630 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:54.000268 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:54.000768 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:54.002521 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:54.004671 (+ 2150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:54.006201 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:54.007346 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:54.007556 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:54.012065 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:54.012077 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:54.012098 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:54.012150 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:54.016452 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:54.017471 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:54.017481 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:54.017491 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:54.017709 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:54.018214 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:54.018230 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:54.018233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:54.018786 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:54.019132 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:54.019649 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 06:55:54.020005 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":344,"threads_started":1}
I20260619 06:55:54.025390 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:54.028436 7267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } }
I20260619 06:55:54.028959 7267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:54.029060 7267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1256b5d3f24352a8e25cf15614ecd6, State: Initialized, Role: FOLLOWER
I20260619 06:55:54.029783 7267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } }
I20260619 06:55:54.030110 7267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:54.030263 7267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:54.030457 7267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:54.033157 7267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } }
I20260619 06:55:54.033823 7267 leader_election.cc:302] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c1256b5d3f24352a8e25cf15614ecd6; no voters:
I20260619 06:55:54.034468 7267 leader_election.cc:288] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:54.034617 7277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:54.036844 7267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:54.037647 7277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [term 1 LEADER]: Becoming Leader. State: Replica: 2c1256b5d3f24352a8e25cf15614ecd6, State: Running, Role: LEADER
I20260619 06:55:54.038808 7277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } }
I20260619 06:55:54.042287 7279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c1256b5d3f24352a8e25cf15614ecd6. Latest consensus state: current_term: 1 leader_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } } }
I20260619 06:55:54.042697 7279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:54.043294 7278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1256b5d3f24352a8e25cf15614ecd6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41007 } } }
I20260619 06:55:54.043617 7278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:54.051440 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:54.030015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:55941 (local address 127.0.0.1:41007)
0619 06:55:54.030214 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:54.030220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:54.030695 (+ 475us) server_negotiation.cc:407] Connection header received
0619 06:55:54.030847 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:54.030856 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:54.030942 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:54.031049 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:54.032457 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:54.033676 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:54.035846 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:54.036804 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:54.036933 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:54.040637 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:54.040646 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:54.040654 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:54.040716 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:54.045606 (+ 4890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:54.046596 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:54.046605 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:54.046612 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:54.046838 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:54.047598 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:54.047606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:54.047612 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:54.048151 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:54.050253 (+ 2102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:54.050497 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 06:55:54.050844 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:55:54.055619 7288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:54.061221 7288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:54.070713 7288 catalog_manager.cc:1347] Generated new cluster ID: c5217552c476438fa68ed85b54deaf90
I20260619 06:55:54.070837 7288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:54.086246 7288 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:54.087775 7288 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:54.098441 7288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c1256b5d3f24352a8e25cf15614ecd6: Generated new TSK 0
I20260619 06:55:54.099754 7288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:54.113463 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7200
I20260619 06:55:54.114804 7262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:55:54.210033 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7200
W20260619 06:55:54.227453 7185 connection.cc:535] client connection to 127.0.0.1:41007 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41007): Connection reset by peer (error 104)
2026-06-19T06:55:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2050 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:54 dist-test-slave-1h5s krb5kdc[7299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:54 dist-test-slave-1h5s krb5kdc[7299](info): set up 2 sockets
Jun 19 06:55:54 dist-test-slave-1h5s krb5kdc[7299](info): commencing operation
krb5kdc: starting...
W20260619 06:55:55.321900 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:55 dist-test-slave-1h5s krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:55.586704 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:55.925549 7312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:55.925858 7312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:55.926038 7312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:55.933439 7312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:55.933590 7312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:55.933674 7312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:55.933765 7312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:55.933843 7312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:55.943980 7312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:55.945395 7312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:55.946700 7312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:55.958395 7317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:55.958317 7318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:55.958352 7320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:55.959821 7312 server_base.cc:1034] running on GCE node
I20260619 06:55:55.960611 7312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:55.962090 7312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:55.963326 7312 hybrid_clock.cc:648] HybridClock initialized: now 1781852155963288 us; error 50 us; skew 500 ppm
I20260619 06:55:55.963670 7312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:55 dist-test-slave-1h5s krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852155, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:55.971132 7312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:55.972779 7312 webserver.cc:460] Webserver started at http://127.0.0.1:36009/ using document root <none> and password file <none>
I20260619 06:55:55.973551 7312 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:55.973693 7312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:55.974005 7312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:55.977025 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "aa5b190a3a714915941a7f1463605e3f"
format_stamp: "Formatted at 2026-06-19 06:55:55 on dist-test-slave-1h5s"
server_key: "86aca5227c28add0099c141ce992d7db"
server_key_iv: "d69fe87d72fc272e2f19b5bb54411c31"
server_key_version: "encryptionkey@0"
I20260619 06:55:55.977844 7312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "aa5b190a3a714915941a7f1463605e3f"
format_stamp: "Formatted at 2026-06-19 06:55:55 on dist-test-slave-1h5s"
server_key: "86aca5227c28add0099c141ce992d7db"
server_key_iv: "d69fe87d72fc272e2f19b5bb54411c31"
server_key_version: "encryptionkey@0"
I20260619 06:55:55.982915 7312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:55:55.986476 7327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:55.987848 7312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:55:55.988025 7312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "aa5b190a3a714915941a7f1463605e3f"
format_stamp: "Formatted at 2026-06-19 06:55:55 on dist-test-slave-1h5s"
server_key: "86aca5227c28add0099c141ce992d7db"
server_key_iv: "d69fe87d72fc272e2f19b5bb54411c31"
server_key_version: "encryptionkey@0"
I20260619 06:55:55.988224 7312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:56.006151 7312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:56.010378 7312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:56.010702 7312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:56.030004 7312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34341
I20260619 06:55:56.030026 7378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34341 every 8 connection(s)
I20260619 06:55:56.031494 7312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:56.035280 7379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:56.041831 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7312
I20260619 06:55:56.042032 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:56.042476 18589 external_mini_cluster.cc:1428] Setting key ac868f08560287fa23b63e36c3b8fdf1
I20260619 06:55:56.046505 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f: Bootstrap starting.
I20260619 06:55:56.050169 7379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:56.051520 7379 log.cc:824] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:56.055154 7379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f: No bootstrap required, opened a new log
I20260619 06:55:56.057123 7382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:56.044448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:40873 (local address 127.0.0.1:34341)
0619 06:55:56.045229 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:56.045241 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:56.045283 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:55:56.047163 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:56.047187 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:56.047593 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:56.047940 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:56.048573 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:56.049976 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:56.051868 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:56.052644 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:56.052816 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:56.054148 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:56.054160 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:56.054179 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:56.054921 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:56.055019 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:56.055062 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:56.055252 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:56.055515 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 06:55:56.055737 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":363,"threads_started":1}
I20260619 06:55:56.061581 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:56.064492 7379 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } }
I20260619 06:55:56.064903 7379 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:56.065032 7379 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa5b190a3a714915941a7f1463605e3f, State: Initialized, Role: FOLLOWER
I20260619 06:55:56.065675 7379 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } }
I20260619 06:55:56.065918 7379 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:56.066082 7379 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:56.066284 7379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:56.069749 7379 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } }
I20260619 06:55:56.070334 7379 leader_election.cc:302] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa5b190a3a714915941a7f1463605e3f; no voters:
I20260619 06:55:56.071002 7379 leader_election.cc:288] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:56.071107 7389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:56.072424 7389 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [term 1 LEADER]: Becoming Leader. State: Replica: aa5b190a3a714915941a7f1463605e3f, State: Running, Role: LEADER
I20260619 06:55:56.073120 7389 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } }
I20260619 06:55:56.073724 7379 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:56.074309 7382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:56.066294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:45501 (local address 127.0.0.1:34341)
0619 06:55:56.066489 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:56.066495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:56.067206 (+ 711us) server_negotiation.cc:407] Connection header received
0619 06:55:56.067380 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:56.067386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:56.067470 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:56.067595 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:56.068078 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:56.069566 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:56.071105 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:56.071903 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:56.072021 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:56.073492 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:56.073503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:56.073507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:56.073525 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:56.073566 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:56.073582 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:56.073699 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:56.073893 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 06:55:56.073967 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:55:56.077242 7390 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa5b190a3a714915941a7f1463605e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } } }
I20260619 06:55:56.077781 7390 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:56.077731 7391 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa5b190a3a714915941a7f1463605e3f. Latest consensus state: current_term: 1 leader_uuid: "aa5b190a3a714915941a7f1463605e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa5b190a3a714915941a7f1463605e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34341 } } }
I20260619 06:55:56.078090 7391 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:56.078820 7396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:56.085000 7396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:56.094513 7396 catalog_manager.cc:1347] Generated new cluster ID: 88409b15e56646b191a09e014ed8df6e
I20260619 06:55:56.094645 7396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:56.119861 7396 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:56.121214 7396 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:56.149892 7396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa5b190a3a714915941a7f1463605e3f: Generated new TSK 0
I20260619 06:55:56.151051 7396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:56.162932 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7312
W20260619 06:55:56.164253 7374 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:40873
W20260619 06:55:56.165143 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7312: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:56.165305 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7312
W20260619 06:55:56.188480 7297 connection.cc:535] client connection to 127.0.0.1:34341 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34341): Connection reset by peer (error 104)
2026-06-19T06:55:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1960 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:56 dist-test-slave-1h5s krb5kdc[7410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:56 dist-test-slave-1h5s krb5kdc[7410](info): set up 2 sockets
Jun 19 06:55:56 dist-test-slave-1h5s krb5kdc[7410](info): commencing operation
krb5kdc: starting...
W20260619 06:55:57.277850 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:57 dist-test-slave-1h5s krb5kdc[7410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:57.536639 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:57.872008 7423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:57.872330 7423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:57.872478 7423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:57.880039 7423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:57.880183 7423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:57.880231 7423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:57.880331 7423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:57.880395 7423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:57.890911 7423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:57.892328 7423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:57.893718 7423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:57.905692 7431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:57.905802 7429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:57.905725 7428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:57.906952 7423 server_base.cc:1034] running on GCE node
I20260619 06:55:57.907801 7423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:57.909260 7423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:57.910511 7423 hybrid_clock.cc:648] HybridClock initialized: now 1781852157910471 us; error 39 us; skew 500 ppm
I20260619 06:55:57.910892 7423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:57 dist-test-slave-1h5s krb5kdc[7410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852157, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:57.914099 7423 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:57.915567 7423 webserver.cc:460] Webserver started at http://127.0.0.1:33263/ using document root <none> and password file <none>
I20260619 06:55:57.916320 7423 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:57.916426 7423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:57.916666 7423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:57.919811 7423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "88914b3c366a480dba1ed94eb086eb2e"
format_stamp: "Formatted at 2026-06-19 06:55:57 on dist-test-slave-1h5s"
server_key: "e7309efdb70a71bfb094fc63b3b20353"
server_key_iv: "36f915b1288569a6f2bdf08ffd2004ae"
server_key_version: "encryptionkey@0"
I20260619 06:55:57.920588 7423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "88914b3c366a480dba1ed94eb086eb2e"
format_stamp: "Formatted at 2026-06-19 06:55:57 on dist-test-slave-1h5s"
server_key: "e7309efdb70a71bfb094fc63b3b20353"
server_key_iv: "36f915b1288569a6f2bdf08ffd2004ae"
server_key_version: "encryptionkey@0"
I20260619 06:55:57.926249 7423 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:55:57.930147 7438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:57.931684 7423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:55:57.931892 7423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "88914b3c366a480dba1ed94eb086eb2e"
format_stamp: "Formatted at 2026-06-19 06:55:57 on dist-test-slave-1h5s"
server_key: "e7309efdb70a71bfb094fc63b3b20353"
server_key_iv: "36f915b1288569a6f2bdf08ffd2004ae"
server_key_version: "encryptionkey@0"
I20260619 06:55:57.932083 7423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:57.938262 7423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:57.942570 7423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:57.942850 7423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:57.962152 7423 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39477
I20260619 06:55:57.962164 7489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39477 every 8 connection(s)
I20260619 06:55:57.963594 7423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:57.967586 7490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:55:57.972277 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7423
I20260619 06:55:57.972478 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:57.972945 18589 external_mini_cluster.cc:1428] Setting key cd1ab4d79d205b959abed64999982979
I20260619 06:55:57.979202 7490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e: Bootstrap starting.
I20260619 06:55:57.981235 7493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:57.974925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:47407 (local address 127.0.0.1:39477)
0619 06:55:57.975788 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:57.975806 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:57.975850 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:55:57.977763 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:57.977791 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:57.978177 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:57.978578 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:57.979636 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:57.979643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:57.979655 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:57.979669 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:57.979738 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:57.979757 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:57.979993 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:57.980130 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 06:55:57.980319 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":347,"threads_started":1}
I20260619 06:55:57.982800 7490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:57.984007 7490 log.cc:824] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:57.985479 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:57.987139 7490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e: No bootstrap required, opened a new log
I20260619 06:55:57.994980 7490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } }
I20260619 06:55:57.995468 7490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:57.995596 7490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88914b3c366a480dba1ed94eb086eb2e, State: Initialized, Role: FOLLOWER
I20260619 06:55:57.996191 7493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:57.990000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:47059 (local address 127.0.0.1:39477)
0619 06:55:57.990205 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:57.990211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:57.994005 (+ 3794us) server_negotiation.cc:407] Connection header received
0619 06:55:57.994156 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:57.994162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:57.994231 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:57.994361 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:57.995338 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:57.995344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:57.995348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:55:57.995372 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:57.995425 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:55:57.995456 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:57.995596 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:57.995721 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 06:55:57.995793 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:55:57.996359 7490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } }
I20260619 06:55:57.996652 7490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:57.996821 7490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:57.997037 7490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:57.999513 7490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } }
I20260619 06:55:58.000025 7490 leader_election.cc:302] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88914b3c366a480dba1ed94eb086eb2e; no voters:
I20260619 06:55:58.000607 7490 leader_election.cc:288] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:58.000759 7500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:58.001953 7500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [term 1 LEADER]: Becoming Leader. State: Replica: 88914b3c366a480dba1ed94eb086eb2e, State: Running, Role: LEADER
I20260619 06:55:58.002595 7500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } }
I20260619 06:55:58.002964 7490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:58.005686 7501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88914b3c366a480dba1ed94eb086eb2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } } }
I20260619 06:55:58.005472 7502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88914b3c366a480dba1ed94eb086eb2e. Latest consensus state: current_term: 1 leader_uuid: "88914b3c366a480dba1ed94eb086eb2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88914b3c366a480dba1ed94eb086eb2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39477 } } }
I20260619 06:55:58.006175 7501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:58.006211 7502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:58.008919 7509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:58.013477 7509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:58.023195 7509 catalog_manager.cc:1347] Generated new cluster ID: 2658e565e29144b495340eaac5cba9ae
I20260619 06:55:58.023335 7509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:58.045904 7509 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:58.046375 7509 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:58.059741 7509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88914b3c366a480dba1ed94eb086eb2e: Generated new TSK 0
I20260619 06:55:58.060701 7509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:58.070809 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7423
W20260619 06:55:58.072132 7485 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:47407
W20260619 06:55:58.072999 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7423: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:55:58.073173 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7423
2026-06-19T06:55:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1900 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:55:58 dist-test-slave-1h5s krb5kdc[7521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:55:58 dist-test-slave-1h5s krb5kdc[7521](info): set up 2 sockets
Jun 19 06:55:58 dist-test-slave-1h5s krb5kdc[7521](info): commencing operation
krb5kdc: starting...
W20260619 06:55:59.181838 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:55:59 dist-test-slave-1h5s krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:55:59.442462 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33467
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:55:59.777984 7534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:55:59.778311 7534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:55:59.778406 7534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:55:59.785722 7534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:55:59.785849 7534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:55:59.785897 7534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:55:59.785949 7534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:55:59.786054 7534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:55:59.796383 7534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:55:59.797708 7534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:55:59.798959 7534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:55:59.811057 7539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:59.810956 7542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:55:59.811024 7540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:55:59.812132 7534 server_base.cc:1034] running on GCE node
I20260619 06:55:59.812924 7534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:55:59.814386 7534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:55:59.815582 7534 hybrid_clock.cc:648] HybridClock initialized: now 1781852159815540 us; error 33 us; skew 500 ppm
I20260619 06:55:59.815927 7534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:55:59 dist-test-slave-1h5s krb5kdc[7521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852159, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:55:59.819060 7534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:55:59.820703 7534 webserver.cc:460] Webserver started at http://127.0.0.1:42975/ using document root <none> and password file <none>
I20260619 06:55:59.821449 7534 fs_manager.cc:359] Metadata directory not provided
I20260619 06:55:59.821587 7534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:55:59.821890 7534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:55:59.824957 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "cbe463832353458d9afa2e3d03223206"
format_stamp: "Formatted at 2026-06-19 06:55:59 on dist-test-slave-1h5s"
server_key: "06e6f0e3e858e1d360730046f26cb03e"
server_key_iv: "8295a87c793692f209b8789859c8b62a"
server_key_version: "encryptionkey@0"
I20260619 06:55:59.825791 7534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "cbe463832353458d9afa2e3d03223206"
format_stamp: "Formatted at 2026-06-19 06:55:59 on dist-test-slave-1h5s"
server_key: "06e6f0e3e858e1d360730046f26cb03e"
server_key_iv: "8295a87c793692f209b8789859c8b62a"
server_key_version: "encryptionkey@0"
I20260619 06:55:59.832012 7534 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 06:55:59.835573 7549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:55:59.836935 7534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:55:59.837131 7534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "cbe463832353458d9afa2e3d03223206"
format_stamp: "Formatted at 2026-06-19 06:55:59 on dist-test-slave-1h5s"
server_key: "06e6f0e3e858e1d360730046f26cb03e"
server_key_iv: "8295a87c793692f209b8789859c8b62a"
server_key_version: "encryptionkey@0"
I20260619 06:55:59.837361 7534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:55:59.860766 7534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:55:59.865288 7534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:55:59.865656 7534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:55:59.884373 7534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33467
I20260619 06:55:59.884403 7600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33467 every 8 connection(s)
I20260619 06:55:59.885809 7534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:55:59.888278 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7534
I20260619 06:55:59.888474 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:55:59.888916 18589 external_mini_cluster.cc:1428] Setting key 2cccdac9c272cbf94a592a6cd8469a14
I20260619 06:55:59.890251 7601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:55:59 dist-test-slave-1h5s krb5kdc[7521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:55:59.904840 7601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206: Bootstrap starting.
I20260619 06:55:59.908438 7601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206: Neither blocks nor log segments found. Creating new log.
I20260619 06:55:59.909597 7601 log.cc:824] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206: Log is configured to *not* fsync() on all Append() calls
I20260619 06:55:59.912623 7601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206: No bootstrap required, opened a new log
I20260619 06:55:59.913264 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:59.890836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:41155 (local address 127.0.0.1:33467)
0619 06:55:59.891625 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:59.891641 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:59.891692 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:55:59.893785 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:59.893813 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:59.894394 (+ 581us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:59.894848 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:59.896300 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:59.898409 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:59.900090 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:59.900897 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:59.901108 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:59.905664 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:59.905672 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:59.905689 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:59.905732 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:59.908964 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:59.909845 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:59.909852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:59.909858 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:59.910019 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:59.910493 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:59.910496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:59.910499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:59.911059 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:59.911384 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:59.911913 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 06:55:59.912219 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":326,"threads_started":1}
I20260619 06:55:59.917588 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:55:59.921792 7601 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260619 06:55:59.922313 7601 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:55:59.922437 7601 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe463832353458d9afa2e3d03223206, State: Initialized, Role: FOLLOWER
I20260619 06:55:59.923108 7601 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260619 06:55:59.923571 7601 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:55:59.923779 7601 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:55:59.923970 7601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:55:59.926375 7601 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260619 06:55:59.926932 7601 leader_election.cc:302] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe463832353458d9afa2e3d03223206; no voters:
I20260619 06:55:59.927613 7601 leader_election.cc:288] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:55:59.927798 7611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:55:59.928557 7611 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [term 1 LEADER]: Becoming Leader. State: Replica: cbe463832353458d9afa2e3d03223206, State: Running, Role: LEADER
I20260619 06:55:59.932248 7611 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } }
I20260619 06:55:59.933944 7601 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:55:59.941619 7604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:55:59.922045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:37699 (local address 127.0.0.1:33467)
0619 06:55:59.922270 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:55:59.922279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:55:59.922859 (+ 580us) server_negotiation.cc:407] Connection header received
0619 06:55:59.923063 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:55:59.923069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:55:59.923151 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:55:59.923253 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:55:59.924897 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:59.925795 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:59.927100 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:55:59.927784 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:55:59.927954 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:55:59.932087 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:55:59.932095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:55:59.932101 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:55:59.932154 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:55:59.936798 (+ 4644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:59.938431 (+ 1633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:59.938439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:59.938446 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:59.938708 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:55:59.939502 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:55:59.939513 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:55:59.939520 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:55:59.940067 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:55:59.940416 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 06:55:59.940755 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 06:55:59.940961 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:55:59.941573 7613 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe463832353458d9afa2e3d03223206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } } }
I20260619 06:55:59.941198 7612 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe463832353458d9afa2e3d03223206. Latest consensus state: current_term: 1 leader_uuid: "cbe463832353458d9afa2e3d03223206" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe463832353458d9afa2e3d03223206" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33467 } } }
I20260619 06:55:59.941905 7613 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:59.941941 7612 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206 [sys.catalog]: This master's current role is: LEADER
I20260619 06:55:59.944103 7621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:55:59.950600 7621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:55:59.961752 7621 catalog_manager.cc:1347] Generated new cluster ID: e1fd8f4e5ac347119b3ed1a16e6639c4
I20260619 06:55:59.961882 7621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:55:59.977572 7621 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:55:59.979426 7621 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:55:59.987289 7621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe463832353458d9afa2e3d03223206: Generated new TSK 0
I20260619 06:55:59.988173 7621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:55:59.995680 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7534
I20260619 06:55:59.997278 7596 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:00.088927 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7534
W20260619 06:56:00.106181 7519 connection.cc:535] client connection to 127.0.0.1:33467 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33467): Connection reset by peer (error 104)
2026-06-19T06:56:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:00 dist-test-slave-1h5s krb5kdc[7633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:00 dist-test-slave-1h5s krb5kdc[7633](info): set up 2 sockets
Jun 19 06:56:00 dist-test-slave-1h5s krb5kdc[7633](info): commencing operation
krb5kdc: starting...
W20260619 06:56:01.210093 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:01 dist-test-slave-1h5s krb5kdc[7633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:56:01.477581 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34229
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:01.816625 7646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:01.816984 7646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:01.817143 7646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:01.824709 7646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:01.824875 7646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:01.824980 7646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:01.825074 7646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:01.825163 7646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:01.835501 7646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34229
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:01.837059 7646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:01.838897 7646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:01.851374 7652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:01.851429 7654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:01.851455 7651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:01.852344 7646 server_base.cc:1034] running on GCE node
I20260619 06:56:01.853142 7646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:01.854734 7646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:01.855993 7646 hybrid_clock.cc:648] HybridClock initialized: now 1781852161855920 us; error 56 us; skew 500 ppm
I20260619 06:56:01.856432 7646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:01 dist-test-slave-1h5s krb5kdc[7633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:01.864586 7646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:56:01.866331 7646 webserver.cc:460] Webserver started at http://127.0.0.1:36549/ using document root <none> and password file <none>
I20260619 06:56:01.867080 7646 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:01.867213 7646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:01.867532 7646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:01.870750 7646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05"
format_stamp: "Formatted at 2026-06-19 06:56:01 on dist-test-slave-1h5s"
server_key: "8dc8c0fa88ceae84163bac866182fb9e"
server_key_iv: "9b9ea1c108af1595e3ac76920bd16b47"
server_key_version: "encryptionkey@0"
I20260619 06:56:01.871582 7646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05"
format_stamp: "Formatted at 2026-06-19 06:56:01 on dist-test-slave-1h5s"
server_key: "8dc8c0fa88ceae84163bac866182fb9e"
server_key_iv: "9b9ea1c108af1595e3ac76920bd16b47"
server_key_version: "encryptionkey@0"
I20260619 06:56:01.877532 7646 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 06:56:01.881042 7661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:01.882695 7646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:01.882938 7646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05"
format_stamp: "Formatted at 2026-06-19 06:56:01 on dist-test-slave-1h5s"
server_key: "8dc8c0fa88ceae84163bac866182fb9e"
server_key_iv: "9b9ea1c108af1595e3ac76920bd16b47"
server_key_version: "encryptionkey@0"
I20260619 06:56:01.883181 7646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:01.906720 7646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:01.912014 7646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:01.912361 7646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:01.932682 7646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34229
I20260619 06:56:01.932724 7712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34229 every 8 connection(s)
I20260619 06:56:01.934367 7646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:01.938261 7713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:01.943944 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7646
I20260619 06:56:01.944149 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:01.944592 18589 external_mini_cluster.cc:1428] Setting key a7e2ead0a2e484ae3c1186ac4ba8d1b4
I20260619 06:56:01.950208 7713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05: Bootstrap starting.
I20260619 06:56:01.954229 7713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:01.955332 7713 log.cc:824] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:01.958489 7713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05: No bootstrap required, opened a new log
I20260619 06:56:01.959893 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:01.946699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38849 (local address 127.0.0.1:34229)
0619 06:56:01.947594 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:01.947609 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:01.947660 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:56:01.950051 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:01.950079 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:01.950641 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:01.951073 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:01.951576 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:01.953681 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:01.955104 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:01.956180 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:01.956413 (+ 233us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:01.957721 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:01.957729 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:01.957744 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:01.958276 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:01.958363 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:01.958392 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:01.958566 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:01.958825 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 06:56:01.959000 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":366,"threads_started":1}
I20260619 06:56:01.964114 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:01.967142 7713 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260619 06:56:01.967545 7713 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:01.967679 7713 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc245a42e1794dfeb6ef6b13a9ee5f05, State: Initialized, Role: FOLLOWER
I20260619 06:56:01.968248 7713 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260619 06:56:01.968490 7713 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:01.968653 7713 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:01.968880 7713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:01.972105 7713 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260619 06:56:01.972635 7713 leader_election.cc:302] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc245a42e1794dfeb6ef6b13a9ee5f05; no voters:
I20260619 06:56:01.973306 7713 leader_election.cc:288] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:01.973551 7723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:01.975834 7713 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:01.977276 7716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:01.968932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:53463 (local address 127.0.0.1:34229)
0619 06:56:01.969118 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:01.969124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:01.969559 (+ 435us) server_negotiation.cc:407] Connection header received
0619 06:56:01.969715 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:01.969720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:01.969808 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:01.969921 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:01.970380 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:01.971529 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:01.973228 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:01.974264 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:01.975293 (+ 1029us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:01.976029 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:01.976041 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:01.976049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:01.976083 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:01.976163 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:01.976194 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:01.976356 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:01.976788 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 06:56:01.976850 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:56:01.980768 7723 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [term 1 LEADER]: Becoming Leader. State: Replica: fc245a42e1794dfeb6ef6b13a9ee5f05, State: Running, Role: LEADER
I20260619 06:56:01.981520 7723 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } }
I20260619 06:56:01.984822 7724 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } } }
I20260619 06:56:01.984903 7725 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc245a42e1794dfeb6ef6b13a9ee5f05. Latest consensus state: current_term: 1 leader_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc245a42e1794dfeb6ef6b13a9ee5f05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34229 } } }
I20260619 06:56:01.985321 7724 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:01.985318 7725 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:01.995126 7733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:02.000375 7733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:02.009791 7733 catalog_manager.cc:1347] Generated new cluster ID: c1f4f769dfd54c61970d605d089be0ac
I20260619 06:56:02.009924 7733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:02.028828 7733 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:02.030776 7733 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:02.039520 7733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc245a42e1794dfeb6ef6b13a9ee5f05: Generated new TSK 0
I20260619 06:56:02.040663 7733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:02.048868 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7646
W20260619 06:56:02.050102 7708 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:38849
W20260619 06:56:02.050851 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7646: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:02.050998 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7646
W20260619 06:56:02.068005 7631 connection.cc:535] client connection to 127.0.0.1:34229 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34229): Connection reset by peer (error 104)
2026-06-19T06:56:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1961 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:02 dist-test-slave-1h5s krb5kdc[7744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:02 dist-test-slave-1h5s krb5kdc[7744](info): set up 2 sockets
Jun 19 06:56:02 dist-test-slave-1h5s krb5kdc[7744](info): commencing operation
krb5kdc: starting...
W20260619 06:56:03.181679 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:03 dist-test-slave-1h5s krb5kdc[7744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:56:03.443846 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:03.781769 7757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:03.782130 7757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:03.782295 7757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:03.789845 7757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:03.789999 7757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:03.790112 7757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:03.790223 7757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:03.790305 7757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:03.800400 7757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:03.801770 7757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:03.803048 7757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:03.814888 7762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:03.814803 7763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:03.814898 7765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:03.815513 7757 server_base.cc:1034] running on GCE node
I20260619 06:56:03.816345 7757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:03.817827 7757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:03.819054 7757 hybrid_clock.cc:648] HybridClock initialized: now 1781852163819009 us; error 37 us; skew 500 ppm
I20260619 06:56:03.819408 7757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:03 dist-test-slave-1h5s krb5kdc[7744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852163, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:03.822891 7757 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:56:03.824301 7757 webserver.cc:460] Webserver started at http://127.0.0.1:39243/ using document root <none> and password file <none>
I20260619 06:56:03.824934 7757 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:03.825028 7757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:03.825284 7757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:03.828291 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "636c67b4fdc0433cab26c9582d0d34d4"
format_stamp: "Formatted at 2026-06-19 06:56:03 on dist-test-slave-1h5s"
server_key: "1852e28d46435b49b7e80ba046282ecd"
server_key_iv: "38bfd62bff04a03aeb89adf842bd0efb"
server_key_version: "encryptionkey@0"
I20260619 06:56:03.829061 7757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "636c67b4fdc0433cab26c9582d0d34d4"
format_stamp: "Formatted at 2026-06-19 06:56:03 on dist-test-slave-1h5s"
server_key: "1852e28d46435b49b7e80ba046282ecd"
server_key_iv: "38bfd62bff04a03aeb89adf842bd0efb"
server_key_version: "encryptionkey@0"
I20260619 06:56:03.834846 7757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:56:03.838845 7772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:03.840193 7757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:56:03.840406 7757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "636c67b4fdc0433cab26c9582d0d34d4"
format_stamp: "Formatted at 2026-06-19 06:56:03 on dist-test-slave-1h5s"
server_key: "1852e28d46435b49b7e80ba046282ecd"
server_key_iv: "38bfd62bff04a03aeb89adf842bd0efb"
server_key_version: "encryptionkey@0"
I20260619 06:56:03.840626 7757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:03.847039 7757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:03.851643 7757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:03.851950 7757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:03.871172 7757 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35901
I20260619 06:56:03.871212 7823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35901 every 8 connection(s)
I20260619 06:56:03.872663 7757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:03.876341 7824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:03.879801 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7757
I20260619 06:56:03.880009 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:03.880465 18589 external_mini_cluster.cc:1428] Setting key 3278c8a76c6971639dc2218a6c0204e7
I20260619 06:56:03.888734 7824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4: Bootstrap starting.
I20260619 06:56:03.890671 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:03.882550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:54709 (local address 127.0.0.1:35901)
0619 06:56:03.883537 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:03.883554 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:03.883931 (+ 377us) server_negotiation.cc:407] Connection header received
0619 06:56:03.886819 (+ 2888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:03.886849 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:03.887300 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:03.887743 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:03.888687 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:03.888694 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:03.888712 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:03.888735 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:03.888799 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:03.888829 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:03.889054 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:03.889174 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 06:56:03.889409 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":423,"threads_started":1}
I20260619 06:56:03.892617 7824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:03.893924 7824 log.cc:824] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:03.894763 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:03.896757 7824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4: No bootstrap required, opened a new log
I20260619 06:56:03.902246 7827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:03.899584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:54289 (local address 127.0.0.1:35901)
0619 06:56:03.899824 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:03.899835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:03.900116 (+ 281us) server_negotiation.cc:407] Connection header received
0619 06:56:03.900306 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:03.900312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:03.900381 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:03.900471 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:03.901485 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:03.901489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:03.901493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:03.901513 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:03.901561 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:03.901577 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:03.901681 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:03.901850 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 06:56:03.901907 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 06:56:03.906002 7824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } }
I20260619 06:56:03.906399 7824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:03.906499 7824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 636c67b4fdc0433cab26c9582d0d34d4, State: Initialized, Role: FOLLOWER
I20260619 06:56:03.907128 7824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } }
I20260619 06:56:03.907366 7824 raft_consensus.cc:388] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:03.907521 7824 raft_consensus.cc:482] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:03.907691 7824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:03.910485 7824 raft_consensus.cc:504] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } }
I20260619 06:56:03.911083 7824 leader_election.cc:302] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 636c67b4fdc0433cab26c9582d0d34d4; no voters:
I20260619 06:56:03.911837 7824 leader_election.cc:288] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:03.911995 7834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:03.913772 7834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [term 1 LEADER]: Becoming Leader. State: Replica: 636c67b4fdc0433cab26c9582d0d34d4, State: Running, Role: LEADER
I20260619 06:56:03.914623 7824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:03.914553 7834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } }
I20260619 06:56:03.920397 7836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 636c67b4fdc0433cab26c9582d0d34d4. Latest consensus state: current_term: 1 leader_uuid: "636c67b4fdc0433cab26c9582d0d34d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } } }
I20260619 06:56:03.920955 7836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:03.927647 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "636c67b4fdc0433cab26c9582d0d34d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "636c67b4fdc0433cab26c9582d0d34d4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35901 } } }
I20260619 06:56:03.927953 7841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:03.928040 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:03.933697 7841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:03.942909 7841 catalog_manager.cc:1347] Generated new cluster ID: d4cac27c28154e849df97043efa321d5
I20260619 06:56:03.943048 7841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:03.967294 7841 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:03.967761 7841 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:03.975086 7841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 636c67b4fdc0433cab26c9582d0d34d4: Generated new TSK 0
I20260619 06:56:03.976194 7841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:03.993711 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7757
W20260619 06:56:03.994912 7819 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:54709
W20260619 06:56:03.995680 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7757: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:03.995837 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7757
2026-06-19T06:56:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1944 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:04 dist-test-slave-1h5s krb5kdc[7855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:04 dist-test-slave-1h5s krb5kdc[7855](info): set up 2 sockets
Jun 19 06:56:04 dist-test-slave-1h5s krb5kdc[7855](info): commencing operation
krb5kdc: starting...
W20260619 06:56:05.109901 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:05 dist-test-slave-1h5s krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:05.377136 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37137
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:37979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:37137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:05.713866 7868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:05.714175 7868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:05.714365 7868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:05.721853 7868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:05.721977 7868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:05.722064 7868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:05.722170 7868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:05.722239 7868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:05.732437 7868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:37979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:37137
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:37137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:05.733999 7868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:05.735334 7868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:05.747308 7873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:05.747357 7874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:05.747419 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:05.748528 7868 server_base.cc:1034] running on GCE node
I20260619 06:56:05.749436 7868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:05.751016 7868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:05.752295 7868 hybrid_clock.cc:648] HybridClock initialized: now 1781852165752244 us; error 55 us; skew 500 ppm
I20260619 06:56:05.752759 7868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:05 dist-test-slave-1h5s krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852165, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:05.756469 7868 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:05.758237 7868 webserver.cc:460] Webserver started at http://127.18.39.126:38211/ using document root <none> and password file <none>
I20260619 06:56:05.759093 7868 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:05.759272 7868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:05.759625 7868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:05.763213 7868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "459092aa8f9a4d09a042e8fddd06cde7"
format_stamp: "Formatted at 2026-06-19 06:56:05 on dist-test-slave-1h5s"
server_key: "9d2fe0c5e016aaf34b7813e2c7b004f2"
server_key_iv: "44b5e178f3a11728a01f04f1e663c2ff"
server_key_version: "encryptionkey@0"
I20260619 06:56:05.764158 7868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "459092aa8f9a4d09a042e8fddd06cde7"
format_stamp: "Formatted at 2026-06-19 06:56:05 on dist-test-slave-1h5s"
server_key: "9d2fe0c5e016aaf34b7813e2c7b004f2"
server_key_iv: "44b5e178f3a11728a01f04f1e663c2ff"
server_key_version: "encryptionkey@0"
I20260619 06:56:05.770258 7868 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260619 06:56:05.774049 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:05.775537 7868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:56:05.775755 7868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "459092aa8f9a4d09a042e8fddd06cde7"
format_stamp: "Formatted at 2026-06-19 06:56:05 on dist-test-slave-1h5s"
server_key: "9d2fe0c5e016aaf34b7813e2c7b004f2"
server_key_iv: "44b5e178f3a11728a01f04f1e663c2ff"
server_key_version: "encryptionkey@0"
I20260619 06:56:05.775981 7868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:05.817664 7868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:05.822652 7868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:05.822992 7868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:05.842677 7868 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:37137
I20260619 06:56:05.842824 7934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:37137 every 8 connection(s)
I20260619 06:56:05.844180 7868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:05.847844 7935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:05.853253 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7868
I20260619 06:56:05.853482 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:05.854064 18589 external_mini_cluster.cc:1428] Setting key b705caefca3c80d9615239c8ed9a2ed8
I20260619 06:56:05.860255 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: Bootstrap starting.
I20260619 06:56:05.864063 7935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:05.865309 7935 log.cc:824] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:05.868851 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: No bootstrap required, opened a new log
Jun 19 06:56:05 dist-test-slave-1h5s krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:56:05.879488 7935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } }
I20260619 06:56:05.880048 7935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:05.880162 7935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 459092aa8f9a4d09a042e8fddd06cde7, State: Initialized, Role: FOLLOWER
I20260619 06:56:05.880843 7935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } }
I20260619 06:56:05.881125 7935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:05.881264 7935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:05.881472 7935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:05.881711 7938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:05.856822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56727 (local address 127.18.39.126:37137)
0619 06:56:05.857806 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:05.857825 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:05.857880 (+ 55us) server_negotiation.cc:407] Connection header received
0619 06:56:05.860271 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:05.860301 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:05.860820 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:05.861254 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:05.862871 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:05.864872 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:05.866601 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:05.867678 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:05.867877 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:05.872269 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:05.872280 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:05.872306 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:05.872375 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:05.876630 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:05.877603 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:05.877619 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:05.877629 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:05.877861 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:05.878382 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:05.878388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:05.878393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:05.879103 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:05.879454 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:05.879902 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 06:56:05.880283 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":368,"threads_started":1}
I20260619 06:56:05.884572 7935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } }
I20260619 06:56:05.885106 7935 leader_election.cc:302] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 459092aa8f9a4d09a042e8fddd06cde7; no voters:
I20260619 06:56:05.885883 7935 leader_election.cc:288] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:05.885951 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:05.886034 7940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:05.887897 7935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:05.893944 7940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [term 1 LEADER]: Becoming Leader. State: Replica: 459092aa8f9a4d09a042e8fddd06cde7, State: Running, Role: LEADER
I20260619 06:56:05.894820 7940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } }
I20260619 06:56:05.899447 7941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "459092aa8f9a4d09a042e8fddd06cde7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } } }
I20260619 06:56:05.899776 7941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:05.900507 7942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 459092aa8f9a4d09a042e8fddd06cde7. Latest consensus state: current_term: 1 leader_uuid: "459092aa8f9a4d09a042e8fddd06cde7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459092aa8f9a4d09a042e8fddd06cde7" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 37137 } } }
I20260619 06:56:05.900825 7942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7 [sys.catalog]: This master's current role is: LEADER
W20260619 06:56:05.906906 7961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:56:05.907050 7961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:56:05.907167 7959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:05.909478 7959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:05.913419 7938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:05.896362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55923 (local address 127.18.39.126:37137)
0619 06:56:05.896601 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:05.896614 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:05.897853 (+ 1239us) server_negotiation.cc:407] Connection header received
0619 06:56:05.898031 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:05.898040 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:05.898146 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:05.898281 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:05.900256 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:05.901707 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:05.903264 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:05.903708 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:05.903875 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:05.906493 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:05.906499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:05.906503 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:05.906538 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:05.910010 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:05.910830 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:05.910836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:05.910840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:05.911034 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:05.911497 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:05.911503 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:05.911506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:05.911908 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:05.912225 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:05.912618 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 06:56:05.913093 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:56:05.919031 7959 catalog_manager.cc:1347] Generated new cluster ID: ad65d1f97cfe476a9a50bf3c8f167d1a
I20260619 06:56:05.919167 7959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:05.934705 7959 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:05.935942 7959 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:05.942935 7959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 459092aa8f9a4d09a042e8fddd06cde7: Generated new TSK 0
I20260619 06:56:05.943902 7959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:05.956620 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7868
I20260619 06:56:05.957857 7930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:06.052511 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7868
W20260619 06:56:06.070127 7853 connection.cc:535] client connection to 127.18.39.126:37137 recv error: Network error: failed to read from TLS socket (remote: 127.18.39.126:37137): Connection reset by peer (error 104)
2026-06-19T06:56:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:06 dist-test-slave-1h5s krb5kdc[7967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:06 dist-test-slave-1h5s krb5kdc[7967](info): set up 2 sockets
Jun 19 06:56:06 dist-test-slave-1h5s krb5kdc[7967](info): commencing operation
krb5kdc: starting...
W20260619 06:56:07.165855 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:07 dist-test-slave-1h5s krb5kdc[7967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:07.425242 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42005
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:46879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:07.762486 7980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:07.762843 7980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:07.763017 7980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:07.770860 7980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:07.770987 7980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:07.771092 7980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:07.771185 7980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:07.771261 7980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:07.781683 7980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:46879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:42005
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:07.783038 7980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:07.784425 7980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:07.796144 7985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:07.796221 7988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:07.796188 7986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:07.797199 7980 server_base.cc:1034] running on GCE node
I20260619 06:56:07.798045 7980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:07.799574 7980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:07.800820 7980 hybrid_clock.cc:648] HybridClock initialized: now 1781852167800774 us; error 72 us; skew 500 ppm
I20260619 06:56:07.801205 7980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:07 dist-test-slave-1h5s krb5kdc[7967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852167, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:07.808550 7980 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:07.810276 7980 webserver.cc:460] Webserver started at http://127.18.39.126:42251/ using document root <none> and password file <none>
I20260619 06:56:07.810972 7980 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:07.811079 7980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:07.811328 7980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:07.814491 7980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "41fa4d9d79e045b487f5b90feb3c5dce"
format_stamp: "Formatted at 2026-06-19 06:56:07 on dist-test-slave-1h5s"
server_key: "0c8855afcd26555a710888ba3c0681ee"
server_key_iv: "8f41c981f44a1aa2cc3266c2ab57a357"
server_key_version: "encryptionkey@0"
I20260619 06:56:07.815249 7980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "41fa4d9d79e045b487f5b90feb3c5dce"
format_stamp: "Formatted at 2026-06-19 06:56:07 on dist-test-slave-1h5s"
server_key: "0c8855afcd26555a710888ba3c0681ee"
server_key_iv: "8f41c981f44a1aa2cc3266c2ab57a357"
server_key_version: "encryptionkey@0"
I20260619 06:56:07.820649 7980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:56:07.824218 7995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:07.825745 7980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:07.825894 7980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "41fa4d9d79e045b487f5b90feb3c5dce"
format_stamp: "Formatted at 2026-06-19 06:56:07 on dist-test-slave-1h5s"
server_key: "0c8855afcd26555a710888ba3c0681ee"
server_key_iv: "8f41c981f44a1aa2cc3266c2ab57a357"
server_key_version: "encryptionkey@0"
I20260619 06:56:07.826064 7980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:07.840155 7980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:07.844470 7980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:07.844707 7980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:07.865641 7980 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:42005
I20260619 06:56:07.865649 8046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:42005 every 8 connection(s)
I20260619 06:56:07.867177 7980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:07.871162 8047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:07.871407 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 7980
I20260619 06:56:07.871598 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:07.872025 18589 external_mini_cluster.cc:1428] Setting key 26a27f85e70c7f705b22a290162cabc4
I20260619 06:56:07.885663 8047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce: Bootstrap starting.
I20260619 06:56:07.886059 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:07.874002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56635 (local address 127.18.39.126:42005)
0619 06:56:07.874867 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:07.874889 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:07.874936 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:56:07.876533 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:07.876556 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:07.877000 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:07.877416 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:07.878055 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:07.879350 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:07.881207 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:07.882002 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:07.882142 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:07.883510 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:07.883517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:07.883532 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:07.884132 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:07.884209 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:07.884239 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:07.884359 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:07.884651 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 06:56:07.884907 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":390,"threads_started":1}
I20260619 06:56:07.889705 8047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:07.890525 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:07.890971 8047 log.cc:824] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:07.894050 8047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce: No bootstrap required, opened a new log
I20260619 06:56:07.902220 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:07.895415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42043 (local address 127.18.39.126:42005)
0619 06:56:07.895626 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:07.895633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:07.896000 (+ 367us) server_negotiation.cc:407] Connection header received
0619 06:56:07.896136 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:07.896141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:07.896227 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:07.896357 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:07.896815 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:07.898052 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:07.899384 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:07.900104 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:07.900238 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:07.901370 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:07.901378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:07.901383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:07.901412 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:07.901472 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:07.901495 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:07.901630 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:07.901786 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 06:56:07.901847 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:56:07.903967 8047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } }
I20260619 06:56:07.904331 8047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:07.904404 8047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41fa4d9d79e045b487f5b90feb3c5dce, State: Initialized, Role: FOLLOWER
I20260619 06:56:07.904985 8047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } }
I20260619 06:56:07.905197 8047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:07.905297 8047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:07.905485 8047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:07.907809 8047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } }
I20260619 06:56:07.908396 8047 leader_election.cc:302] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41fa4d9d79e045b487f5b90feb3c5dce; no voters:
I20260619 06:56:07.908917 8047 leader_election.cc:288] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:07.909080 8057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:07.909855 8057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [term 1 LEADER]: Becoming Leader. State: Replica: 41fa4d9d79e045b487f5b90feb3c5dce, State: Running, Role: LEADER
I20260619 06:56:07.910475 8057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } }
I20260619 06:56:07.911089 8047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:07.913583 8058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } } }
I20260619 06:56:07.914042 8058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:07.913926 8059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41fa4d9d79e045b487f5b90feb3c5dce. Latest consensus state: current_term: 1 leader_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41fa4d9d79e045b487f5b90feb3c5dce" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 42005 } } }
I20260619 06:56:07.914206 8059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:07.915762 8064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:07.921022 8064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:07.929972 8064 catalog_manager.cc:1347] Generated new cluster ID: 2f13b62e065940f890741f30f92951a9
I20260619 06:56:07.930130 8064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:07.950526 8064 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:07.952533 8064 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:07.961205 8064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41fa4d9d79e045b487f5b90feb3c5dce: Generated new TSK 0
I20260619 06:56:07.962113 8064 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:07.968185 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7980
W20260619 06:56:07.969450 8042 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56635
W20260619 06:56:07.970225 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 7980: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:07.970376 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 7980
W20260619 06:56:07.987048 7965 connection.cc:535] client connection to 127.18.39.126:42005 recv error: Network error: failed to read from TLS socket (remote: 127.18.39.126:42005): Connection reset by peer (error 104)
2026-06-19T06:56:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1916 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:08 dist-test-slave-1h5s krb5kdc[8078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:08 dist-test-slave-1h5s krb5kdc[8078](info): set up 2 sockets
Jun 19 06:56:08 dist-test-slave-1h5s krb5kdc[8078](info): commencing operation
krb5kdc: starting...
W20260619 06:56:09.085826 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:09 dist-test-slave-1h5s krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:09.344547 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38593
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:36991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:38593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:09.682170 8091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:09.682478 8091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:09.682570 8091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:09.689992 8091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:09.690127 8091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:09.690176 8091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:09.690239 8091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:09.690285 8091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:09.700491 8091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:36991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:38593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:09.701931 8091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:09.703238 8091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:09.715620 8096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:09.715680 8097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:09.715720 8099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:09.716605 8091 server_base.cc:1034] running on GCE node
I20260619 06:56:09.717496 8091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:09.718978 8091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:09.720211 8091 hybrid_clock.cc:648] HybridClock initialized: now 1781852169720160 us; error 55 us; skew 500 ppm
I20260619 06:56:09.720570 8091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:09 dist-test-slave-1h5s krb5kdc[8078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852169, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:09.723889 8091 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:09.725522 8091 webserver.cc:460] Webserver started at http://127.18.39.126:42963/ using document root <none> and password file <none>
I20260619 06:56:09.726255 8091 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:09.726405 8091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:09.726709 8091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:09.729908 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037"
format_stamp: "Formatted at 2026-06-19 06:56:09 on dist-test-slave-1h5s"
server_key: "18f01a2afe177674b2894e59a0cb2bbe"
server_key_iv: "a74506210956f0070b430872e54fd367"
server_key_version: "encryptionkey@0"
I20260619 06:56:09.730710 8091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037"
format_stamp: "Formatted at 2026-06-19 06:56:09 on dist-test-slave-1h5s"
server_key: "18f01a2afe177674b2894e59a0cb2bbe"
server_key_iv: "a74506210956f0070b430872e54fd367"
server_key_version: "encryptionkey@0"
I20260619 06:56:09.736249 8091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 06:56:09.739953 8106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:09.741425 8091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:56:09.741618 8091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037"
format_stamp: "Formatted at 2026-06-19 06:56:09 on dist-test-slave-1h5s"
server_key: "18f01a2afe177674b2894e59a0cb2bbe"
server_key_iv: "a74506210956f0070b430872e54fd367"
server_key_version: "encryptionkey@0"
I20260619 06:56:09.741844 8091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:09.747552 8091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:09.751869 8091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:09.752194 8091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:09.771377 8091 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:38593
I20260619 06:56:09.771410 8157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:38593 every 8 connection(s)
I20260619 06:56:09.772964 8091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:09.776814 8158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:09.780380 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8091
I20260619 06:56:09.780571 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:09.781044 18589 external_mini_cluster.cc:1428] Setting key 32da3000d43d5c5e98a364738ae10194
I20260619 06:56:09.788755 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:09.782870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59937 (local address 127.18.39.126:38593)
0619 06:56:09.783616 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:09.783628 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:09.783667 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:56:09.785313 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:09.785397 (+ 84us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:09.785765 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:09.786164 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:09.787193 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:09.787203 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:09.787215 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:09.787228 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:09.787280 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:09.787301 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:09.787535 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:09.787669 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 06:56:09.787843 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260619 06:56:09.789041 8158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037: Bootstrap starting.
I20260619 06:56:09.792495 8158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:09.792562 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:09.793610 8158 log.cc:824] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:09.796710 8158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037: No bootstrap required, opened a new log
I20260619 06:56:09.799593 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:09.797027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46803 (local address 127.18.39.126:38593)
0619 06:56:09.797213 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:09.797221 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:09.797605 (+ 384us) server_negotiation.cc:407] Connection header received
0619 06:56:09.797812 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:09.797816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:09.797865 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:09.797969 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:09.798934 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:09.798946 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:09.798950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:09.798969 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:09.799006 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:09.799020 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:09.799148 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:09.799240 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 06:56:09.799307 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:56:09.804961 8158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } }
I20260619 06:56:09.805505 8158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:09.805632 8158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744f05ef7cc04d3e9d3ff6b8fd6de037, State: Initialized, Role: FOLLOWER
I20260619 06:56:09.806265 8158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } }
I20260619 06:56:09.806568 8158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:09.806727 8158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:09.806941 8158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:09.809521 8158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } }
I20260619 06:56:09.810170 8158 leader_election.cc:302] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 744f05ef7cc04d3e9d3ff6b8fd6de037; no voters:
I20260619 06:56:09.810887 8158 leader_election.cc:288] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:09.810993 8168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:09.811615 8168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [term 1 LEADER]: Becoming Leader. State: Replica: 744f05ef7cc04d3e9d3ff6b8fd6de037, State: Running, Role: LEADER
I20260619 06:56:09.812242 8168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } }
I20260619 06:56:09.813261 8158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:09.815140 8169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } } }
I20260619 06:56:09.815641 8169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:09.817363 8175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:09.817477 8170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744f05ef7cc04d3e9d3ff6b8fd6de037. Latest consensus state: current_term: 1 leader_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744f05ef7cc04d3e9d3ff6b8fd6de037" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38593 } } }
I20260619 06:56:09.817796 8170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:09.822443 8175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:09.831696 8175 catalog_manager.cc:1347] Generated new cluster ID: 9f5571636185466baba65f02cba62016
I20260619 06:56:09.831838 8175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:09.864228 8175 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:09.864698 8175 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:09.878387 8175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 744f05ef7cc04d3e9d3ff6b8fd6de037: Generated new TSK 0
I20260619 06:56:09.879313 8175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:09.891773 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8091
W20260619 06:56:09.892953 8153 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59937
W20260619 06:56:09.893723 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8091: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:09.893878 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8091
2026-06-19T06:56:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1923 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:09 dist-test-slave-1h5s krb5kdc[8189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:09 dist-test-slave-1h5s krb5kdc[8189](info): set up 2 sockets
Jun 19 06:56:09 dist-test-slave-1h5s krb5kdc[8189](info): commencing operation
krb5kdc: starting...
W20260619 06:56:11.010021 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:11 dist-test-slave-1h5s krb5kdc[8189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:11.268813 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35347
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:35347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:11.606030 8202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:11.606362 8202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:11.606532 8202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:11.614050 8202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:11.614197 8202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:11.614310 8202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:11.614418 8202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:11.614483 8202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:11.624665 8202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:35347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:35347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:11.626071 8202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:11.627380 8202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:11.639266 8208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:11.639328 8210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:11.639349 8207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:11.640208 8202 server_base.cc:1034] running on GCE node
I20260619 06:56:11.641045 8202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:11.642598 8202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:11.643850 8202 hybrid_clock.cc:648] HybridClock initialized: now 1781852171643806 us; error 53 us; skew 500 ppm
I20260619 06:56:11.644235 8202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:11 dist-test-slave-1h5s krb5kdc[8189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852171, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:11.647644 8202 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:11.649309 8202 webserver.cc:460] Webserver started at http://127.18.39.126:33421/ using document root <none> and password file <none>
I20260619 06:56:11.650058 8202 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:11.650159 8202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:11.650414 8202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:11.653448 8202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "97efecfca9b4466db9d6119501872fa2"
format_stamp: "Formatted at 2026-06-19 06:56:11 on dist-test-slave-1h5s"
server_key: "4584c580456ed4d5b2da03f219fad800"
server_key_iv: "a60661f9332148af6fdde1ec23b8fba7"
server_key_version: "encryptionkey@0"
I20260619 06:56:11.654163 8202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "97efecfca9b4466db9d6119501872fa2"
format_stamp: "Formatted at 2026-06-19 06:56:11 on dist-test-slave-1h5s"
server_key: "4584c580456ed4d5b2da03f219fad800"
server_key_iv: "a60661f9332148af6fdde1ec23b8fba7"
server_key_version: "encryptionkey@0"
I20260619 06:56:11.659260 8202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:56:11.662855 8217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:11.664278 8202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:56:11.664490 8202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "97efecfca9b4466db9d6119501872fa2"
format_stamp: "Formatted at 2026-06-19 06:56:11 on dist-test-slave-1h5s"
server_key: "4584c580456ed4d5b2da03f219fad800"
server_key_iv: "a60661f9332148af6fdde1ec23b8fba7"
server_key_version: "encryptionkey@0"
I20260619 06:56:11.664700 8202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:11.684762 8202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:11.689240 8202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:11.689581 8202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:11.708350 8202 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:35347
I20260619 06:56:11.708371 8268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:35347 every 8 connection(s)
I20260619 06:56:11.709815 8202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:11.713639 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8202
I20260619 06:56:11.713828 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:11.713994 8269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:11.714406 18589 external_mini_cluster.cc:1428] Setting key 6faeefaa6f44feff98f029d833d0f22a
I20260619 06:56:11.728818 8269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2: Bootstrap starting.
Jun 19 06:56:11 dist-test-slave-1h5s krb5kdc[8189](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:56:11.732304 8269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:11.733803 8269 log.cc:824] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:11.737265 8269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2: No bootstrap required, opened a new log
I20260619 06:56:11.740753 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:11.716501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58851 (local address 127.18.39.126:35347)
0619 06:56:11.717555 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:11.717571 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:11.717970 (+ 399us) server_negotiation.cc:407] Connection header received
0619 06:56:11.720732 (+ 2762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:11.720765 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:11.721386 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:11.721896 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:11.723214 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:11.725253 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:11.726534 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:11.727583 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:11.727755 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:11.732198 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:11.732212 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:11.732232 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:11.732300 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:11.736207 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:11.736905 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:11.736914 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:11.736922 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:11.737129 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:11.737566 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:11.737574 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:11.737579 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:11.738296 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:11.738602 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:11.738887 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 06:56:11.739341 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":449,"threads_started":1}
I20260619 06:56:11.744632 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:11.747431 8269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } }
I20260619 06:56:11.748008 8269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:11.748184 8269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97efecfca9b4466db9d6119501872fa2, State: Initialized, Role: FOLLOWER
I20260619 06:56:11.748894 8269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } }
I20260619 06:56:11.749217 8269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:11.749420 8269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:11.749619 8269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:11.752682 8269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } }
I20260619 06:56:11.753392 8269 leader_election.cc:302] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97efecfca9b4466db9d6119501872fa2; no voters:
I20260619 06:56:11.754068 8269 leader_election.cc:288] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:11.754210 8279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:11.756328 8269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:11.756894 8279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [term 1 LEADER]: Becoming Leader. State: Replica: 97efecfca9b4466db9d6119501872fa2, State: Running, Role: LEADER
I20260619 06:56:11.757659 8279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } }
I20260619 06:56:11.761204 8280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97efecfca9b4466db9d6119501872fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } } }
I20260619 06:56:11.761826 8280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:11.761997 8281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97efecfca9b4466db9d6119501872fa2. Latest consensus state: current_term: 1 leader_uuid: "97efecfca9b4466db9d6119501872fa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97efecfca9b4466db9d6119501872fa2" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 35347 } } }
I20260619 06:56:11.766155 8281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:11.766943 8288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:11.770288 8272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:11.749168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32829 (local address 127.18.39.126:35347)
0619 06:56:11.749419 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:11.749427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:11.749954 (+ 527us) server_negotiation.cc:407] Connection header received
0619 06:56:11.750105 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:11.750110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:11.750200 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:11.750362 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:11.751721 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:11.752933 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:11.754533 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:11.755526 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:11.755657 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:11.759271 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:11.759281 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:11.759289 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:11.759336 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:11.767290 (+ 7954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:11.768021 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:11.768031 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:11.768035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:11.768173 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:11.768592 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:11.768597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:11.768600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:11.769044 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:11.769352 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:11.769712 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:56:11.769933 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:56:11.774833 8288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:11.784682 8288 catalog_manager.cc:1347] Generated new cluster ID: 998e0e3a9e3845fba4ed20f96b313e93
I20260619 06:56:11.784825 8288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:11.797765 8288 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:11.799634 8288 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:11.809278 8288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97efecfca9b4466db9d6119501872fa2: Generated new TSK 0
I20260619 06:56:11.810453 8288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:11.819762 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8202
I20260619 06:56:11.820880 8264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:11.916195 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8202
2026-06-19T06:56:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2022 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:12 dist-test-slave-1h5s krb5kdc[8301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:12 dist-test-slave-1h5s krb5kdc[8301](info): set up 2 sockets
Jun 19 06:56:12 dist-test-slave-1h5s krb5kdc[8301](info): commencing operation
krb5kdc: starting...
W20260619 06:56:13.025960 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:13 dist-test-slave-1h5s krb5kdc[8301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:13.293022 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38511
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:44641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:38511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:13.630659 8314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:13.630981 8314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:13.631153 8314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:13.638602 8314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:13.638731 8314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:13.638845 8314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:13.638957 8314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:13.639045 8314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:13.649212 8314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:44641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:38511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:38511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:13.650650 8314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:13.652000 8314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:13.664418 8319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:13.664968 8320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:13.665036 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:13.665555 8314 server_base.cc:1034] running on GCE node
I20260619 06:56:13.666410 8314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:13.668083 8314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:13.669478 8314 hybrid_clock.cc:648] HybridClock initialized: now 1781852173669418 us; error 45 us; skew 500 ppm
I20260619 06:56:13.669902 8314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:13 dist-test-slave-1h5s krb5kdc[8301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852173, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:13.682287 8314 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:13.684152 8314 webserver.cc:460] Webserver started at http://127.18.39.126:39127/ using document root <none> and password file <none>
I20260619 06:56:13.684916 8314 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:13.685034 8314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:13.685294 8314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:13.689414 8314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec"
format_stamp: "Formatted at 2026-06-19 06:56:13 on dist-test-slave-1h5s"
server_key: "7ecd823d2f7e1774294eabfea49605c3"
server_key_iv: "010378ca18a5805f6b35d3942f5fcb0d"
server_key_version: "encryptionkey@0"
I20260619 06:56:13.690337 8314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec"
format_stamp: "Formatted at 2026-06-19 06:56:13 on dist-test-slave-1h5s"
server_key: "7ecd823d2f7e1774294eabfea49605c3"
server_key_iv: "010378ca18a5805f6b35d3942f5fcb0d"
server_key_version: "encryptionkey@0"
I20260619 06:56:13.696290 8314 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260619 06:56:13.700227 8329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:13.701875 8314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:13.702107 8314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec"
format_stamp: "Formatted at 2026-06-19 06:56:13 on dist-test-slave-1h5s"
server_key: "7ecd823d2f7e1774294eabfea49605c3"
server_key_iv: "010378ca18a5805f6b35d3942f5fcb0d"
server_key_version: "encryptionkey@0"
I20260619 06:56:13.702329 8314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:13.715835 8314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:13.720398 8314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:13.720731 8314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:13.740374 8314 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:38511
I20260619 06:56:13.740407 8380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:38511 every 8 connection(s)
I20260619 06:56:13.741928 8314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:13.746537 8381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:13.749143 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8314
I20260619 06:56:13.749372 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:13.749892 18589 external_mini_cluster.cc:1428] Setting key 54e7a81705543d5e036481d48ebc2fe9
I20260619 06:56:13.760361 8381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec: Bootstrap starting.
I20260619 06:56:13.763953 8381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:13.764833 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:13.751951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47643 (local address 127.18.39.126:38511)
0619 06:56:13.752910 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:13.752927 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:13.752990 (+ 63us) server_negotiation.cc:407] Connection header received
0619 06:56:13.754895 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:13.754923 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:13.755364 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:13.755766 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:13.756509 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:13.758397 (+ 1888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:13.759781 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:13.760634 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:13.760859 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:13.762291 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:13.762300 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:13.762322 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:13.762882 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:13.762966 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:13.763000 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:13.763143 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:13.763389 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 06:56:13.763669 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":409,"threads_started":1}
I20260619 06:56:13.765165 8381 log.cc:824] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:13.768237 8381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec: No bootstrap required, opened a new log
I20260619 06:56:13.768821 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:13.777614 8381 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } }
I20260619 06:56:13.778141 8381 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:13.778264 8381 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfb6ac79e1a4eaaa2e1fb124ca62aec, State: Initialized, Role: FOLLOWER
I20260619 06:56:13.779006 8381 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } }
I20260619 06:56:13.779225 8381 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:13.779418 8381 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:13.779610 8381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:13.780925 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:13.773834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56967 (local address 127.18.39.126:38511)
0619 06:56:13.774063 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:13.774071 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:13.774478 (+ 407us) server_negotiation.cc:407] Connection header received
0619 06:56:13.774614 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:13.774620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:13.774701 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:13.774812 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:13.775304 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:13.776500 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:13.777872 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:13.778569 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:13.778699 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:13.780106 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:13.780114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:13.780120 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:13.780143 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:13.780198 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:13.780221 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:13.780328 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:13.780464 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 06:56:13.780563 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:56:13.782569 8381 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } }
I20260619 06:56:13.783113 8381 leader_election.cc:302] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcfb6ac79e1a4eaaa2e1fb124ca62aec; no voters:
I20260619 06:56:13.783704 8381 leader_election.cc:288] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:13.783931 8391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:13.784709 8391 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [term 1 LEADER]: Becoming Leader. State: Replica: bcfb6ac79e1a4eaaa2e1fb124ca62aec, State: Running, Role: LEADER
I20260619 06:56:13.785482 8391 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } }
I20260619 06:56:13.786192 8381 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:13.788842 8392 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } } }
I20260619 06:56:13.789242 8392 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:13.789664 8393 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcfb6ac79e1a4eaaa2e1fb124ca62aec. Latest consensus state: current_term: 1 leader_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfb6ac79e1a4eaaa2e1fb124ca62aec" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 38511 } } }
I20260619 06:56:13.790032 8393 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:13.794167 8397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:13.799885 8397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:13.810021 8397 catalog_manager.cc:1347] Generated new cluster ID: d5d1b3192a7747b799d091931c8457ee
I20260619 06:56:13.810169 8397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:13.831794 8397 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:13.833830 8397 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:13.843636 8397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcfb6ac79e1a4eaaa2e1fb124ca62aec: Generated new TSK 0
I20260619 06:56:13.844560 8397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:13.865576 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8314
W20260619 06:56:13.866679 8376 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47643
W20260619 06:56:13.867419 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8314: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:13.867574 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8314
2026-06-19T06:56:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1951 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:13 dist-test-slave-1h5s krb5kdc[8412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:13 dist-test-slave-1h5s krb5kdc[8412](info): set up 2 sockets
Jun 19 06:56:13 dist-test-slave-1h5s krb5kdc[8412](info): commencing operation
krb5kdc: starting...
W20260619 06:56:14.982057 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:15 dist-test-slave-1h5s krb5kdc[8412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:15.250468 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36431
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:36431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:15.592010 8425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:15.592340 8425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:15.592514 8425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:15.600157 8425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:15.600288 8425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:15.600374 8425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:15.600478 8425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:15.600567 8425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:15.611179 8425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:36431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:36431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:15.612574 8425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:15.614001 8425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:15.626959 8431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:15.627007 8430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:15.627632 8425 server_base.cc:1034] running on GCE node
W20260619 06:56:15.627707 8433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:15.628729 8425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:15.630479 8425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:15.631757 8425 hybrid_clock.cc:648] HybridClock initialized: now 1781852175631668 us; error 59 us; skew 500 ppm
I20260619 06:56:15.632187 8425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:15 dist-test-slave-1h5s krb5kdc[8412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852175, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:15.635879 8425 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:15.637657 8425 webserver.cc:460] Webserver started at http://127.18.39.126:46439/ using document root <none> and password file <none>
I20260619 06:56:15.638397 8425 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:15.638505 8425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:15.638864 8425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:15.642124 8425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "e8563a957b5e4c158528f8df56d73f88"
format_stamp: "Formatted at 2026-06-19 06:56:15 on dist-test-slave-1h5s"
server_key: "5820ff113901240b0650f0ced0fd2fb8"
server_key_iv: "2a56e0421d60898405a360deaf20fb4a"
server_key_version: "encryptionkey@0"
I20260619 06:56:15.642946 8425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "e8563a957b5e4c158528f8df56d73f88"
format_stamp: "Formatted at 2026-06-19 06:56:15 on dist-test-slave-1h5s"
server_key: "5820ff113901240b0650f0ced0fd2fb8"
server_key_iv: "2a56e0421d60898405a360deaf20fb4a"
server_key_version: "encryptionkey@0"
I20260619 06:56:15.648690 8425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:56:15.652370 8440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:15.653963 8425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:15.654165 8425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "e8563a957b5e4c158528f8df56d73f88"
format_stamp: "Formatted at 2026-06-19 06:56:15 on dist-test-slave-1h5s"
server_key: "5820ff113901240b0650f0ced0fd2fb8"
server_key_iv: "2a56e0421d60898405a360deaf20fb4a"
server_key_version: "encryptionkey@0"
I20260619 06:56:15.654376 8425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:15.660703 8425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:15.665108 8425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:15.665506 8425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:15.685106 8425 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:36431
I20260619 06:56:15.685141 8491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:36431 every 8 connection(s)
I20260619 06:56:15.686657 8425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:15.690550 8492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:15.696939 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8425
I20260619 06:56:15.697177 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:15.697690 18589 external_mini_cluster.cc:1428] Setting key 720ad53b132b0e212c7adae4fad70592
I20260619 06:56:15.702009 8492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88: Bootstrap starting.
I20260619 06:56:15.705519 8492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:15.705919 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:15.699592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49155 (local address 127.18.39.126:36431)
0619 06:56:15.700436 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:15.700448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:15.700490 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:56:15.702371 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:15.702394 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:15.702749 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:15.703136 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:15.704226 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:15.704232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:15.704245 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:15.704258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:15.704322 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:15.704342 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:15.704571 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:15.704704 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 06:56:15.704886 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":375,"threads_started":1}
I20260619 06:56:15.706882 8492 log.cc:824] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:15.709930 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:15.710014 8492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88: No bootstrap required, opened a new log
I20260619 06:56:15.716961 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:15.714410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47699 (local address 127.18.39.126:36431)
0619 06:56:15.714602 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:15.714608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:15.714982 (+ 374us) server_negotiation.cc:407] Connection header received
0619 06:56:15.715128 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:15.715133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:15.715196 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:15.715315 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:15.716241 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:15.716247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:15.716252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:56:15.716271 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:15.716320 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:56:15.716338 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:15.716459 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:15.716538 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 06:56:15.716610 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:56:15.718860 8492 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } }
I20260619 06:56:15.719249 8492 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:56:15.719374 8492 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8563a957b5e4c158528f8df56d73f88, State: Initialized, Role: FOLLOWER
I20260619 06:56:15.719981 8492 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } }
I20260619 06:56:15.720224 8492 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:56:15.720384 8492 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:56:15.720564 8492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:15.723266 8492 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } }
I20260619 06:56:15.723814 8492 leader_election.cc:302] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8563a957b5e4c158528f8df56d73f88; no voters:
I20260619 06:56:15.724498 8492 leader_election.cc:288] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:56:15.724609 8502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:15.725307 8502 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [term 1 LEADER]: Becoming Leader. State: Replica: e8563a957b5e4c158528f8df56d73f88, State: Running, Role: LEADER
I20260619 06:56:15.726008 8502 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } }
I20260619 06:56:15.727147 8492 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:15.729202 8504 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8563a957b5e4c158528f8df56d73f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } } }
I20260619 06:56:15.729782 8503 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8563a957b5e4c158528f8df56d73f88. Latest consensus state: current_term: 1 leader_uuid: "e8563a957b5e4c158528f8df56d73f88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8563a957b5e4c158528f8df56d73f88" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 36431 } } }
I20260619 06:56:15.730100 8503 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:15.729681 8504 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:15.737936 8512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:15.742224 8512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:15.751242 8512 catalog_manager.cc:1347] Generated new cluster ID: 0d425fb018c0425ba122e5e67b57aeb0
I20260619 06:56:15.751377 8512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:15.791762 8512 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:15.792323 8512 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:15.802937 8512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8563a957b5e4c158528f8df56d73f88: Generated new TSK 0
I20260619 06:56:15.803838 8512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:15.820569 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8425
W20260619 06:56:15.821749 8487 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49155
W20260619 06:56:15.822500 18589 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8425: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:56:15.822670 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8425
2026-06-19T06:56:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1954 ms)
[----------] 18 tests from AuthTokenIssuingTest (35526 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:15 dist-test-slave-1h5s krb5kdc[8523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:15 dist-test-slave-1h5s krb5kdc[8523](info): set up 2 sockets
Jun 19 06:56:15 dist-test-slave-1h5s krb5kdc[8523](info): commencing operation
krb5kdc: starting...
W20260619 06:56:16.942140 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:17 dist-test-slave-1h5s krb5kdc[8523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:17.208751 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39333
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:17.549187 8536 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:17.549690 8536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:17.549827 8536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:17.549973 8536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:17.557687 8536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:17.557826 8536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:17.557943 8536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:17.558041 8536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:17.558131 8536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:17.568755 8536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:39333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:17.570211 8536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:17.571522 8536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:17.583355 8544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:17.583426 8541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:17.583395 8542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:17.584491 8536 server_base.cc:1034] running on GCE node
I20260619 06:56:17.585379 8536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:17.586859 8536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:17.588115 8536 hybrid_clock.cc:648] HybridClock initialized: now 1781852177588050 us; error 47 us; skew 500 ppm
I20260619 06:56:17.588472 8536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:17 dist-test-slave-1h5s krb5kdc[8523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:17.591830 8536 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:17.593632 8536 webserver.cc:460] Webserver started at http://127.18.39.126:34571/ using document root <none> and password file <none>
I20260619 06:56:17.594410 8536 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:17.594548 8536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:17.594887 8536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:17.598198 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "3d9255b20ace424580e790e2628caa16"
format_stamp: "Formatted at 2026-06-19 06:56:17 on dist-test-slave-1h5s"
server_key: "3c48b834b62dbd627729f4f805591a03"
server_key_iv: "3a08c4f1c145420c4c06d6f82359800a"
server_key_version: "encryptionkey@0"
I20260619 06:56:17.599093 8536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "3d9255b20ace424580e790e2628caa16"
format_stamp: "Formatted at 2026-06-19 06:56:17 on dist-test-slave-1h5s"
server_key: "3c48b834b62dbd627729f4f805591a03"
server_key_iv: "3a08c4f1c145420c4c06d6f82359800a"
server_key_version: "encryptionkey@0"
I20260619 06:56:17.604899 8536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260619 06:56:17.609143 8551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:17.610882 8536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:17.611138 8536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "3d9255b20ace424580e790e2628caa16"
format_stamp: "Formatted at 2026-06-19 06:56:17 on dist-test-slave-1h5s"
server_key: "3c48b834b62dbd627729f4f805591a03"
server_key_iv: "3a08c4f1c145420c4c06d6f82359800a"
server_key_version: "encryptionkey@0"
I20260619 06:56:17.611416 8536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:17.625239 8536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:17.629981 8536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:17.630283 8536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:17.649621 8536 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:39333
I20260619 06:56:17.649628 8602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:39333 every 8 connection(s)
I20260619 06:56:17.651113 8536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:17.655131 8603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:17.655207 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8536
I20260619 06:56:17.655411 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:17.656090 18589 external_mini_cluster.cc:1428] Setting key 1662921e9c0797485d03ded22f733029
I20260619 06:56:17.662506 8603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:17 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.125@KRBTEST.COM" created.
I20260619 06:56:17.690676 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.666035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48244 (local address 127.18.39.126:39333)
0619 06:56:17.666932 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:17.666943 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:17.667844 (+ 901us) server_negotiation.cc:407] Connection header received
0619 06:56:17.671149 (+ 3305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:17.671155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:17.671418 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:17.671587 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:17.673406 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:17.674624 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:17.676924 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:17.677591 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:17.677730 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:17.682014 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:17.682029 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:17.682052 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:17.682101 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:17.685973 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:17.686778 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:17.686785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:17.686787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:17.686970 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:17.687388 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:17.687391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:17.687393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:17.688169 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:17.688338 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:17.689160 (+ 822us) server_negotiation.cc:299] Negotiation successful
0619 06:56:17.689556 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":688,"thread_start_us":427,"threads_started":1}
I20260619 06:56:17.690716 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.666044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39333 (local address 127.0.0.1:48244)
0619 06:56:17.667658 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.667704 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:17.670697 (+ 2993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:17.671678 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:17.671688 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:17.672555 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:17.673098 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:17.673126 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:17.674791 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:17.674800 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:17.676736 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:17.676754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:17.677839 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:17.677860 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:17.678089 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:17.678840 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:17.678886 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:17.681727 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:17.686118 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:17.686129 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:17.686140 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:17.686657 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:17.687075 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:17.687078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:17.687080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:17.687290 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:17.688365 (+ 1075us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:17.688375 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:17.688692 (+ 317us) client_negotiation.cc:769] Sending connection context
0619 06:56:17.689048 (+ 356us) client_negotiation.cc:240] Negotiation successful
0619 06:56:17.689762 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1342,"mutex_wait_us":122,"thread_start_us":441,"threads_started":1}
I20260619 06:56:17.693845 8603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:17.694952 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.694558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33530)
0619 06:56:17.694724 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.694831 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 06:56:17.695302 8553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
I20260619 06:56:17.696925 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.696555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33536)
0619 06:56:17.696703 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.696777 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 06:56:17.697264 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:43789: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
I20260619 06:56:17.719784 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:43789
--webserver_interface=127.18.39.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:56:17.746721 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } attempt: 1
I20260619 06:56:17.748026 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.747469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33544)
0619 06:56:17.747703 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.747851 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260619 06:56:17.749212 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.748714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33548)
0619 06:56:17.748897 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.748998 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 06:56:17.749493 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:43789: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
I20260619 06:56:17.823457 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } attempt: 2
I20260619 06:56:17.825824 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.824288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33564)
0619 06:56:17.825531 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.825645 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1077}
I20260619 06:56:17.826900 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.826479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33580)
0619 06:56:17.826641 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.826741 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 06:56:17.827131 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:43789: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
I20260619 06:56:17.918385 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } attempt: 3
I20260619 06:56:17.919703 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.919058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33596)
0619 06:56:17.919330 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.919491 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:56:17.921031 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:17.920485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33604)
0619 06:56:17.920696 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:17.920812 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260619 06:56:17.921312 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:43789: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
W20260619 06:56:18.063957 8610 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:18.064311 8610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:18.064379 8610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:18.064466 8610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 06:56:18.064638 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } attempt: 4
I20260619 06:56:18.065934 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.065306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33608)
0619 06:56:18.065658 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.065767 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260619 06:56:18.067063 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.066642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33620)
0619 06:56:18.066796 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.066905 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 06:56:18.067255 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:43789: Network error: Client connection negotiation failed: client connection to 127.18.39.125:43789: connect: Connection refused (error 111)
W20260619 06:56:18.073100 8610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:18.073213 8610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:18.073274 8610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:18.073319 8610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:18.073457 8610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:18.083880 8610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:43789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.39.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:18.085222 8610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:18.086674 8610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:18.099026 8615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:18.099037 8616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:18.099076 8618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:18.100170 8610 server_base.cc:1034] running on GCE node
I20260619 06:56:18.101040 8610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:18.102690 8610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:18.103942 8610 hybrid_clock.cc:648] HybridClock initialized: now 1781852178103888 us; error 40 us; skew 500 ppm
I20260619 06:56:18.104300 8610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:18.107698 8610 init.cc:377] Logged in from keytab as kudu/127.18.39.125@KRBTEST.COM (short username kudu)
I20260619 06:56:18.109423 8610 webserver.cc:460] Webserver started at http://127.18.39.125:45527/ using document root <none> and password file <none>
I20260619 06:56:18.110234 8610 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:18.110386 8610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:18.110728 8610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:18.113988 8610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data/instance:
uuid: "b3e99dd571b648c7b3dfaeddc062698b"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "7b801900bf353d4824d4e5f6e9cdf707"
server_key_iv: "823141cc510442d4e7fdb1486ffe0f9c"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.114931 8610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal/instance:
uuid: "b3e99dd571b648c7b3dfaeddc062698b"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "7b801900bf353d4824d4e5f6e9cdf707"
server_key_iv: "823141cc510442d4e7fdb1486ffe0f9c"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.120713 8610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:56:18.124666 8625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:18.126173 8610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:18.126398 8610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "b3e99dd571b648c7b3dfaeddc062698b"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "7b801900bf353d4824d4e5f6e9cdf707"
server_key_iv: "823141cc510442d4e7fdb1486ffe0f9c"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.126662 8610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:18.139411 8610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:18.144443 8610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:18.144754 8610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:18.164175 8610 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.125:43789
I20260619 06:56:18.164193 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.125:43789 every 8 connection(s)
I20260619 06:56:18.165719 8610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
I20260619 06:56:18.169831 8677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:18.171011 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8610
I20260619 06:56:18.171218 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-1/wal/instance
I20260619 06:56:18.171772 18589 external_mini_cluster.cc:1428] Setting key 51aa332a951f17620efecfdcc3e7dd2d
I20260619 06:56:18.180614 8677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.124@KRBTEST.COM" created.
I20260619 06:56:18.206341 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.183940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.18.39.126:39333)
0619 06:56:18.184179 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.184187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.185410 (+ 1223us) server_negotiation.cc:407] Connection header received
0619 06:56:18.188502 (+ 3092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.188506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.188584 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.188673 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.190976 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.191667 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.194224 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.194661 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.194763 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.199280 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.199286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.199289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.199326 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.202565 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.203384 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.203390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.203393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.203531 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.204071 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.204079 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.204085 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.204511 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.204683 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.205603 (+ 920us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.205898 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:56:18.207216 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.184392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39333 (local address 127.0.0.1:48252)
0619 06:56:18.185204 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.185255 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.188134 (+ 2879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.188970 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.188984 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.189842 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.190709 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.190736 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.191847 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.191855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.194033 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.194052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.194907 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.194918 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.195283 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.196136 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.196172 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.198989 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.202733 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.202743 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.202764 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.203234 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.203648 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.203653 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.203657 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.203911 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.204658 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.204668 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.205139 (+ 471us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.205445 (+ 306us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.206009 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":306,"threads_started":1}
I20260619 06:56:18.209185 8677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:18.226253 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.209945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33626)
0619 06:56:18.210253 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.210274 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.210384 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.211283 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.211291 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.211838 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.212079 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.212094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.213911 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.213915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.214777 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.214787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.216071 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.216076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.216283 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.216908 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.216924 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.219261 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.223793 (+ 4532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.223797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.223799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.224166 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.224588 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.224591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.224594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.224732 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.225523 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.225527 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.225641 (+ 114us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.225751 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.225985 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":39}
I20260619 06:56:18.226742 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.210152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.18.39.125:43789)
0619 06:56:18.210848 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.210856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.210881 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:56:18.210929 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.210935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.211111 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.211230 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.212256 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.213739 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.214914 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.215916 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.216062 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.219497 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.219515 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.219538 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.219582 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.223647 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.224300 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.224312 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.224318 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.224479 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.224838 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.224845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.224848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.225389 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.225598 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.225945 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.226239 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":403,"threads_started":1}
I20260619 06:56:18.228962 8677 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:18.230159 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.229673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39562)
0619 06:56:18.229881 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.230006 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:56:18.230409 8628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.232085 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.231725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39564)
0619 06:56:18.231883 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.231980 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:56:18.232299 8677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
I20260619 06:56:18.238682 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:34601
--webserver_interface=127.18.39.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:56:18.281648 8677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 1
I20260619 06:56:18.282842 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.282387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39578)
0619 06:56:18.282586 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.282692 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 06:56:18.283795 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.283453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39594)
0619 06:56:18.283623 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.283693 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 06:56:18.283998 8677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.352241 8677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 2
I20260619 06:56:18.353724 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.353083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39600)
0619 06:56:18.353386 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.353509 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 06:56:18.355065 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.354613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39604)
0619 06:56:18.354804 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.354886 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 06:56:18.355417 8677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.366523 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } attempt: 5
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:18.381428 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.367489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.18.39.125:43789)
0619 06:56:18.367666 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.367671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.367690 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:56:18.367797 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.367801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.367883 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.368006 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.368947 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.369703 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.371350 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.371796 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.371941 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.375781 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.375787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.375790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.375824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.378692 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.379561 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.379566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.379568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.379686 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.380084 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.380087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.380089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.380364 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.380476 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.380957 (+ 481us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.381128 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 06:56:18.381551 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.367280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33634)
0619 06:56:18.367514 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.367537 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.367678 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.368054 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.368058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.368572 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.368796 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.368808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.369835 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.369840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.371181 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.371207 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.371982 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.371988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.372182 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.372842 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.372866 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.375564 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.378870 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.378875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.378880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.379437 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.379797 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.379802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.379806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.379980 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.380495 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.380503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.380704 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.380860 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.381156 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 06:56:18.382756 8603 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:18.384037 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.383577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39618)
0619 06:56:18.383773 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.383866 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 06:56:18.385156 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.384722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39626)
0619 06:56:18.384901 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.385016 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 06:56:18.385390 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.436657 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 1
I20260619 06:56:18.438035 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.437412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39640)
0619 06:56:18.437663 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.437779 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 06:56:18.439267 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.438839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39650)
0619 06:56:18.439025 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.439131 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 06:56:18.439510 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.446707 8677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 3
I20260619 06:56:18.447894 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.447383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39652)
0619 06:56:18.447594 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.447702 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:56:18.449002 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.448565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39662)
0619 06:56:18.448813 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.448884 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260619 06:56:18.449213 8677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.507761 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 2
I20260619 06:56:18.509074 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.508570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39672)
0619 06:56:18.508800 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.508916 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:56:18.510155 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.509802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39680)
0619 06:56:18.509958 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.510041 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 06:56:18.510345 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
W20260619 06:56:18.585567 8684 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:18.586009 8684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:18.586139 8684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:18.586282 8684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 06:56:18.592427 8677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 4
I20260619 06:56:18.593647 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.593123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39690)
0619 06:56:18.593403 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.593498 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 06:56:18.594028 8684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:18.594147 8684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:18.594272 8684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:18.594363 8684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:18.594460 8684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:18.594658 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.594309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39700)
0619 06:56:18.594449 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.594521 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 06:56:18.594852 8677 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
I20260619 06:56:18.605055 8684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:39333,127.18.39.125:43789,127.18.39.124:34601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:34601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.39.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:18.606478 8684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:18.607810 8684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 06:56:18.616537 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 3
I20260619 06:56:18.617810 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.617172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39714)
0619 06:56:18.617469 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.617583 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 06:56:18.618964 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.618500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39720)
0619 06:56:18.618714 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.618824 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260619 06:56:18.619207 8603 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:34601: Network error: Client connection negotiation failed: client connection to 127.18.39.124:34601: connect: Connection refused (error 111)
W20260619 06:56:18.622717 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:18.622766 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:18.622841 8689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:18.624084 8684 server_base.cc:1034] running on GCE node
I20260619 06:56:18.624974 8684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:18.626586 8684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:18.627851 8684 hybrid_clock.cc:648] HybridClock initialized: now 1781852178627778 us; error 44 us; skew 500 ppm
I20260619 06:56:18.628222 8684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:18.631678 8684 init.cc:377] Logged in from keytab as kudu/127.18.39.124@KRBTEST.COM (short username kudu)
I20260619 06:56:18.633258 8684 webserver.cc:460] Webserver started at http://127.18.39.124:35135/ using document root <none> and password file <none>
I20260619 06:56:18.634027 8684 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:18.634116 8684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:18.634389 8684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:18.637291 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data/instance:
uuid: "f5575c44f96542caa79844a15fbcfce6"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "65fd8532ec428169c59dd153f866226f"
server_key_iv: "96073802793e5849325193c2162632b8"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.638129 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal/instance:
uuid: "f5575c44f96542caa79844a15fbcfce6"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "65fd8532ec428169c59dd153f866226f"
server_key_iv: "96073802793e5849325193c2162632b8"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.643293 8684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:56:18.646875 8699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:18.648320 8684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:56:18.648514 8684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "f5575c44f96542caa79844a15fbcfce6"
format_stamp: "Formatted at 2026-06-19 06:56:18 on dist-test-slave-1h5s"
server_key: "65fd8532ec428169c59dd153f866226f"
server_key_iv: "96073802793e5849325193c2162632b8"
server_key_version: "encryptionkey@0"
I20260619 06:56:18.648751 8684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:18.669510 8684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:18.673895 8684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:18.674173 8684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:18.693253 8684 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.124:34601
I20260619 06:56:18.693272 8750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.124:34601 every 8 connection(s)
I20260619 06:56:18.694808 8684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
I20260619 06:56:18.698567 8751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:18.704438 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8684
I20260619 06:56:18.704641 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851818886250-18589-0/minicluster-data/master-2/wal/instance
I20260619 06:56:18.705070 18589 external_mini_cluster.cc:1428] Setting key 4fd7af18c668ab43efb7fb79d24c0845
I20260619 06:56:18.705276 8751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:56:18.730816 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.708123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.18.39.126:39333)
0619 06:56:18.708656 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.708662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.709233 (+ 571us) server_negotiation.cc:407] Connection header received
0619 06:56:18.710638 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.710642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.710708 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.710795 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.712281 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.713137 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.714937 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.715610 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.715738 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.723844 (+ 8106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.723852 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.723855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.723896 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.727325 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.728119 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.728125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.728130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.728261 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.728696 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.728701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.728706 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.728993 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.729144 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.730110 (+ 966us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.730409 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":266,"threads_started":1}
I20260619 06:56:18.731860 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.708152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:39333 (local address 127.0.0.1:48264)
0619 06:56:18.709043 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.709103 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.710352 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.710886 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.710893 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.711696 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.712084 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.712102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.713314 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.713324 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.714807 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.714819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.715910 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.715917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.716097 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.720853 (+ 4756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.720881 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.723575 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.727506 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.727521 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.727535 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.727976 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.728374 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.728381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.728386 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.728576 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.729135 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.729143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.729661 (+ 518us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.729969 (+ 308us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.730537 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":49,"spinlock_wait_cycles":18816,"thread_start_us":462,"threads_started":1}
I20260619 06:56:18.731876 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.707196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.18.39.124:34601)
0619 06:56:18.708007 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.708021 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.708080 (+ 59us) server_negotiation.cc:407] Connection header received
0619 06:56:18.710352 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.710379 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.710685 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.710900 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.712439 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.713935 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.715373 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.716093 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.716235 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.724295 (+ 8060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.724305 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.724321 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.724359 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.727680 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.728433 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.728449 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.728451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.728586 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.728962 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.728971 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.728973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.729652 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.729993 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.730320 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.730625 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":580,"thread_start_us":370,"threads_started":1}
I20260619 06:56:18.733582 8751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:56:18.748366 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.734235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:43789 (local address 127.0.0.1:33650)
0619 06:56:18.734475 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.734489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.734581 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.734951 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.734956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.735367 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.735519 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.735527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.736559 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.736565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.737666 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.737678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.738405 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.738411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.738624 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.739388 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.739406 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.741690 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.745773 (+ 4083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.745778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.745781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.746232 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.746693 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.746696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.746699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.746876 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.747590 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.747594 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.747740 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.747898 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.748122 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260619 06:56:18.748750 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.734311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.18.39.125:43789)
0619 06:56:18.734516 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.734523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.734549 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:56:18.734674 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.734679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.734770 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.734858 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.735662 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.736413 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.737815 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.738244 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.738349 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.741942 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.741951 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.741961 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.742020 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.745616 (+ 3596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.746368 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.746376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.746381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.746564 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.746993 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.747001 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.747005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.747436 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.747620 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.748043 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.748306 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:56:18.749852 8751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:18.753309 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.735122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.18.39.126:39333)
0619 06:56:18.735300 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.735305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.735322 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:56:18.735500 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.735505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.735611 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.735708 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.736954 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.737706 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.739208 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.739740 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.739905 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.745140 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.745149 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.745155 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.745206 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.749042 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.749978 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.749983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.749985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.750156 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.750620 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.750625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.750629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.751469 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.751769 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.752712 (+ 943us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.752980 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:56:18.764212 8756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.750754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39732)
0619 06:56:18.750928 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.750948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.751045 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.751292 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.751295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.751638 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.751840 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.751855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.752811 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.752816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.753941 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.753953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.754612 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.754617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.754880 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.755514 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.755536 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.757813 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.761253 (+ 3440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.761259 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.761264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.761849 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.762214 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.762221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.762224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.762416 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.763219 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.763224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.763457 (+ 233us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.763620 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.763941 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:18.764694 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.750907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39732 (local address 127.18.39.124:34601)
0619 06:56:18.751067 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.751072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.751086 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:56:18.751130 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.751133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.751186 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.751261 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.751991 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.752680 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.754073 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.754469 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.754605 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.758012 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.758022 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.758029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.758076 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.761101 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.761972 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.761976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.761978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.762101 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.762564 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.762573 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.762576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.763083 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.763250 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.763718 (+ 468us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.764138 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":162,"server-negotiator.queue_time_us":37}
I20260619 06:56:18.771183 8751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: Bootstrap starting.
I20260619 06:56:18.772324 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.755026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.18.39.125:43789)
0619 06:56:18.755545 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.755552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.755571 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:56:18.755631 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.755636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.755713 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.755820 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.757065 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.757984 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.759339 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.760040 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.760181 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.765656 (+ 5475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.765666 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.765673 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.765723 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.769119 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.770100 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.770107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.770111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.770307 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.770747 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.770753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.770756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.771125 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.771433 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.771759 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.772012 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260619 06:56:18.775549 8751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:18.776914 8751 log.cc:824] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:18.777876 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:18.780287 8751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: No bootstrap required, opened a new log
I20260619 06:56:18.781543 8578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d9255b20ace424580e790e2628caa16"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48272
I20260619 06:56:18.787940 8751 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.788314 8751 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5575c44f96542caa79844a15fbcfce6, State: Initialized, Role: FOLLOWER
I20260619 06:56:18.788782 8751 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.789866 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:18.790329 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:18.790911 8751 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:18.796428 8603 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 4
W20260619 06:56:18.801896 8769 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:56:18.802062 8769 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852177, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:56:18.799710 8684 master_runner.cc:186] Error getting master registration for 127.18.39.126:39333: OK, instance_id { permanent_uuid: "3d9255b20ace424580e790e2628caa16" instance_seqno: 1781852177644163 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:56:18.811764 8684 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:56:18.816468 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.797228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39744)
0619 06:56:18.797448 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.797472 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.797603 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.798289 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.798294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.798840 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.799096 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.799110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.800848 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.800857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.802424 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.802439 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.803757 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.803764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.803990 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.804720 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.804748 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.807946 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.813184 (+ 5238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.813191 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.813195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.813752 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.814222 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.814227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.814231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.814425 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.815240 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.815246 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.815508 (+ 262us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.815673 (+ 165us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.816014 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 06:56:18.816469 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.797688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.18.39.124:34601)
0619 06:56:18.797929 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.797936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.797958 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:56:18.798022 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.798028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.798111 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.798231 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.799254 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.800561 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.802597 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.803582 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.803727 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.808251 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.808262 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.808276 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.808350 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.812993 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.813915 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.813922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.813926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.814074 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.814588 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.814593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.814596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.815071 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.815296 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.815801 (+ 505us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.816037 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:56:18.823282 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: Bootstrap starting.
I20260619 06:56:18.827651 8603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:18.829042 8603 log.cc:824] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:18.832911 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: No bootstrap required, opened a new log
I20260619 06:56:18.841076 8603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.841607 8603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d9255b20ace424580e790e2628caa16, State: Initialized, Role: FOLLOWER
I20260619 06:56:18.842171 8603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.843255 8772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:18.843690 8772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:18.844666 8603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:18.844720 8578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 06:56:18.844959 8578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:18.848361 8578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.851274 8726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d9255b20ace424580e790e2628caa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f5575c44f96542caa79844a15fbcfce6"
I20260619 06:56:18.851846 8726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:18.855734 8726 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9255b20ace424580e790e2628caa16 in term 1.
I20260619 06:56:18.856597 8554 leader_election.cc:302] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d9255b20ace424580e790e2628caa16, f5575c44f96542caa79844a15fbcfce6; no voters:
I20260619 06:56:18.857134 8772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:18.857231 8578 leader_election.cc:288] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [CANDIDATE]: Term 1 election: Requested vote from peers b3e99dd571b648c7b3dfaeddc062698b (127.18.39.125:43789), f5575c44f96542caa79844a15fbcfce6 (127.18.39.124:34601)
I20260619 06:56:18.859388 8772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [term 1 LEADER]: Becoming Leader. State: Replica: 3d9255b20ace424580e790e2628caa16, State: Running, Role: LEADER
I20260619 06:56:18.860113 8772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:18.864780 8783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d9255b20ace424580e790e2628caa16. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:18.865199 8783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: This master's current role is: LEADER
W20260619 06:56:18.883239 8557 tablet.cc:2369] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:56:18.884740 8787 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:56:18.884953 8787 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:56:18.885170 8786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:18.887912 8786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:18.871166 8536 master_runner.cc:186] Error getting master registration for 127.18.39.125:43789: OK, instance_id { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" instance_seqno: 1781852178158755 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:56:18.889724 8536 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:56:18.894039 8677 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } attempt: 5
Jun 19 06:56:18 dist-test-slave-1h5s krb5kdc[8523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852178, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
W20260619 06:56:18.926429 8705 tablet.cc:2369] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:56:18.926741 8726 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [term 1 FOLLOWER]: Refusing update from remote peer 3d9255b20ace424580e790e2628caa16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:56:18.927876 8783 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:56:18.938134 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.896894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.18.39.126:39333)
0619 06:56:18.897059 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.897065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.910865 (+ 13800us) server_negotiation.cc:407] Connection header received
0619 06:56:18.912854 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.912865 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.912985 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.913073 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.916381 (+ 3308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.917864 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.920211 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.920928 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.921041 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.925434 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.925442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.925446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.925498 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.933283 (+ 7785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.934917 (+ 1634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.934923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.934928 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.935108 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.935863 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.935869 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.935873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.936272 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.936563 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.937457 (+ 894us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.937742 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:56:18.938110 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.895085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.18.39.124:34601)
0619 06:56:18.895259 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.895264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.895286 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:56:18.895444 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.895449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.895556 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.895653 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.897032 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.898217 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.900458 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.901208 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.901359 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.923497 (+ 22138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.923506 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.923511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.923562 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.931026 (+ 7464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.932641 (+ 1615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.932669 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.932676 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.933019 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:18.934018 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:18.934027 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:18.934062 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:18.935121 (+ 1059us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:18.935402 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:18.936544 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0619 06:56:18.937139 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:56:18.953168 8680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.894726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:34601 (local address 127.0.0.1:39756)
0619 06:56:18.895158 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:18.895178 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:18.895319 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:18.895686 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:18.895691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:18.896231 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:18.896469 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.896487 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.898360 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.898370 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.900297 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:18.900319 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.901426 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.901434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:18.901919 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.902990 (+ 1071us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:18.903049 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:18.921244 (+ 18195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:18.931224 (+ 9980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.931230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.931233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.932409 (+ 1176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.933303 (+ 894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:18.933312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:18.933368 (+ 56us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:18.933772 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:18.935598 (+ 1826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:18.935607 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:18.935985 (+ 378us) client_negotiation.cc:769] Sending connection context
0619 06:56:18.949367 (+ 13382us) client_negotiation.cc:240] Negotiation successful
0619 06:56:18.951029 (+ 1662us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297}
I20260619 06:56:18.964665 8677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: Bootstrap starting.
I20260619 06:56:18.984290 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d9255b20ace424580e790e2628caa16. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:18.984733 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:18.988600 8677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:18.990170 8677 log.cc:824] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:19.003057 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.911604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.18.39.125:43789)
0619 06:56:18.911856 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.911865 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.940035 (+ 28170us) server_negotiation.cc:407] Connection header received
0619 06:56:18.940442 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.940448 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.940528 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.940601 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.953555 (+ 12954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.954708 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.967755 (+ 13047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.968789 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.968942 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:18.991973 (+ 23031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:18.991983 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:18.991988 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:18.992045 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:18.996175 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:19.000160 (+ 3985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:19.000168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:19.000174 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:19.000405 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:19.000957 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:19.000963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:19.000967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:19.001442 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:19.001715 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.002369 (+ 654us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.002661 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 06:56:19.006750 8790 mvcc.cc:205] Tried to move back new op lower bound from 7298466524871573504 to 7298466524618907648. Current Snapshot: MvccSnapshot[applied={T|T < 7298466524871573504}]
I20260619 06:56:19.012483 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:18.907001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.18.39.124:34601)
0619 06:56:18.939041 (+ 32040us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:18.939049 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:18.939099 (+ 50us) server_negotiation.cc:407] Connection header received
0619 06:56:18.939224 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:18.939235 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:18.939442 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:18.942459 (+ 3017us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:18.946272 (+ 3813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.948529 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.981082 (+ 32553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:18.982225 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:18.982389 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.004453 (+ 22064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:19.004461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:19.004467 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:19.004512 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:19.008633 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:19.009643 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:19.009650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:19.009654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:19.009814 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:19.010273 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:19.010278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:19.010282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:19.010663 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:19.010913 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.011541 (+ 628us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.011788 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31754,"thread_start_us":39145,"threads_started":1}
I20260619 06:56:19.016621 8783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.016971 8783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:19.017930 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.018203 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:19.019093 8786 catalog_manager.cc:1347] Generated new cluster ID: 69a7fc7ad1cd489699f9f9429f23592e
I20260619 06:56:19.019186 8786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:19.023854 8677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: No bootstrap required, opened a new log
I20260619 06:56:19.035769 8677 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:19.036358 8677 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e99dd571b648c7b3dfaeddc062698b, State: Initialized, Role: FOLLOWER
I20260619 06:56:19.036989 8677 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } }
I20260619 06:56:19.044162 8802 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.046723 8802 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:19.046568 8677 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:19.047044 8651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:19.046593 8652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d9255b20ace424580e790e2628caa16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b3e99dd571b648c7b3dfaeddc062698b"
I20260619 06:56:19.052692 8786 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:19.054747 8786 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:19.056078 8651 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [term 1 FOLLOWER]: Refusing update from remote peer 3d9255b20ace424580e790e2628caa16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:56:19.059695 8652 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d9255b20ace424580e790e2628caa16 in term 1.
I20260619 06:56:19.062201 8772 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:56:19.081692 8786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16: Generated new TSK 0
I20260619 06:56:19.083307 8786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:19.116266 8802 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d9255b20ace424580e790e2628caa16. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.116683 8802 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:19.117201 8809 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.117628 8809 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:19.119094 8783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d9255b20ace424580e790e2628caa16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d9255b20ace424580e790e2628caa16" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 39333 } } peers { permanent_uuid: "b3e99dd571b648c7b3dfaeddc062698b" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 43789 } } peers { permanent_uuid: "f5575c44f96542caa79844a15fbcfce6" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 34601 } } }
I20260619 06:56:19.119380 8783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d9255b20ace424580e790e2628caa16 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:19.144487 8818 catalog_manager.cc:1259] Loaded cluster ID: 69a7fc7ad1cd489699f9f9429f23592e
I20260619 06:56:19.144681 8818 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: loading cluster ID for follower catalog manager: success
I20260619 06:56:19.148981 8818 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: acquiring CA information for follower catalog manager: success
I20260619 06:56:19.151777 8818 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b3e99dd571b648c7b3dfaeddc062698b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 06:56:19.241683 8755 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.218632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.18.39.126:39333)
0619 06:56:19.218847 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.218857 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.239194 (+ 20337us) server_negotiation.cc:407] Connection header received
0619 06:56:19.239834 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.239842 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.239943 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.240025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.241398 (+ 1373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48300: BlockingRecv error: recv got EOF from 127.0.0.1:48300 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260619 06:56:19.244797 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.224836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39770 (local address 127.18.39.124:34601)
0619 06:56:19.225035 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.225042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.242724 (+ 17682us) server_negotiation.cc:407] Connection header received
0619 06:56:19.243037 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.243044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.243139 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.243229 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.244487 (+ 1258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39770: BlockingRecv error: recv got EOF from 127.0.0.1:39770 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260619 06:56:19.249171 8681 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.246430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33666 (local address 127.18.39.125:43789)
0619 06:56:19.246678 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.246686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.247404 (+ 718us) server_negotiation.cc:407] Connection header received
0619 06:56:19.247672 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.247678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.247796 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.247879 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.248921 (+ 1042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33666: BlockingRecv error: recv got EOF from 127.0.0.1:33666 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 06:56:19.306185 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.286946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.18.39.125:43789)
0619 06:56:19.287142 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.287148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.297927 (+ 10779us) server_negotiation.cc:407] Connection header received
0619 06:56:19.298517 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.298524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.298745 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.298894 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.299659 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.300833 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.302975 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.303679 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.303785 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.304612 (+ 827us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.305313 (+ 701us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.305462 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.305718 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.305821 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 06:56:19.346526 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.336530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39774 (local address 127.18.39.124:34601)
0619 06:56:19.336758 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.336765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.344227 (+ 7462us) server_negotiation.cc:407] Connection header received
0619 06:56:19.344791 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.344797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.344939 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.345030 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.346236 (+ 1206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39774: BlockingRecv error: recv got EOF from 127.0.0.1:39774 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:56:19.405184 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.387414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.18.39.125:43789)
0619 06:56:19.387597 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.387604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.397810 (+ 10206us) server_negotiation.cc:407] Connection header received
0619 06:56:19.398366 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.398373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.398492 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.398560 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.399248 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.400398 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.402669 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.403440 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.403544 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.404174 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.404363 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.404523 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.404759 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.404854 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 06:56:19.480242 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.459707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39784 (local address 127.18.39.124:34601)
0619 06:56:19.459994 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.460002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.478114 (+ 18112us) server_negotiation.cc:407] Connection header received
0619 06:56:19.478474 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.478481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.478603 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.478692 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.479940 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39784: BlockingRecv error: recv got EOF from 127.0.0.1:39784 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 06:56:19.529561 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.517778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.18.39.125:43789)
0619 06:56:19.517998 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.518004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.522457 (+ 4453us) server_negotiation.cc:407] Connection header received
0619 06:56:19.522689 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.522696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.522798 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.522936 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.523479 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.524719 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.527104 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.527818 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.527997 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.528558 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.528744 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.528854 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.529062 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.529167 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 06:56:19.540395 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.538297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.18.39.124:34601)
0619 06:56:19.538490 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.538496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.538661 (+ 165us) server_negotiation.cc:407] Connection header received
0619 06:56:19.539012 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.539018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.539129 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.539246 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.540141 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39790: BlockingRecv error: recv got EOF from 127.0.0.1:39790 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:56:19.559201 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.551448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.18.39.125:43789)
0619 06:56:19.551706 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.551733 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.552161 (+ 428us) server_negotiation.cc:407] Connection header received
0619 06:56:19.552419 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.552428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.552530 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.552644 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.553173 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.554413 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.556669 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.557436 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.557569 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.558192 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.558343 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.558441 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.558615 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.558704 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260619 06:56:19.569747 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.567829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.18.39.124:34601)
0619 06:56:19.568126 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.568134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.568159 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:56:19.568250 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.568257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.568364 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.568484 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.569485 (+ 1001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39796: BlockingRecv error: recv got EOF from 127.0.0.1:39796 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
I20260619 06:56:19.589172 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.581458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.18.39.125:43789)
0619 06:56:19.581677 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.581683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.582284 (+ 601us) server_negotiation.cc:407] Connection header received
0619 06:56:19.582503 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.582512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.582618 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.582741 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.583304 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.584589 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.586618 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.587475 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.587620 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.588121 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.588305 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.588408 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.588576 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.588675 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260619 06:56:19.604301 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.596674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39802 (local address 127.18.39.124:34601)
0619 06:56:19.596889 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.596895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.602131 (+ 5236us) server_negotiation.cc:407] Connection header received
0619 06:56:19.602372 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.602378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.602481 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.602590 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.604057 (+ 1467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39802: BlockingRecv error: recv got EOF from 127.0.0.1:39802 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:56:19.629999 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.623632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.18.39.125:43789)
0619 06:56:19.623866 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.623873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.624195 (+ 322us) server_negotiation.cc:407] Connection header received
0619 06:56:19.624374 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.624381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.624479 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.624610 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.624970 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.626278 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.627792 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.628542 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.628717 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.629051 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.629212 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.629308 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.629518 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.629616 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 06:56:19.638442 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.636356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.18.39.124:34601)
0619 06:56:19.636575 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.636581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.636918 (+ 337us) server_negotiation.cc:407] Connection header received
0619 06:56:19.637115 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.637119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.637205 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.637290 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.638213 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39818: BlockingRecv error: recv got EOF from 127.0.0.1:39818 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:56:19.680739 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.674544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.18.39.125:43789)
0619 06:56:19.674817 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.674823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.675118 (+ 295us) server_negotiation.cc:407] Connection header received
0619 06:56:19.675320 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.675324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.675412 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.675491 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.675990 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.676735 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.678786 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.679307 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.679408 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.679931 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.680063 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.680129 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.680320 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.680387 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260619 06:56:19.689097 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.686947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.18.39.124:34601)
0619 06:56:19.687190 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.687195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.687542 (+ 347us) server_negotiation.cc:407] Connection header received
0619 06:56:19.687753 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.687757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.687836 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.687931 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.688857 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39832: BlockingRecv error: recv got EOF from 127.0.0.1:39832 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:56:19.763067 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.756998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.18.39.125:43789)
0619 06:56:19.757268 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.757276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.757644 (+ 368us) server_negotiation.cc:407] Connection header received
0619 06:56:19.757920 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.757924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.757997 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.758106 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.758548 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.759293 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.761284 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.761721 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.761887 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.762307 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.762433 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.762534 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.762675 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.762759 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260619 06:56:19.771090 8754 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:19.769293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.18.39.124:34601)
0619 06:56:19.769517 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.769521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.769825 (+ 304us) server_negotiation.cc:407] Connection header received
0619 06:56:19.769994 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.769998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.770079 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.770169 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:19.770869 (+ 700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39838: BlockingRecv error: recv got EOF from 127.0.0.1:39838 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 06:56:19.804064 8769 catalog_manager.cc:1259] Loaded cluster ID: 69a7fc7ad1cd489699f9f9429f23592e
I20260619 06:56:19.804219 8769 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: loading cluster ID for follower catalog manager: success
I20260619 06:56:19.806913 8769 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: acquiring CA information for follower catalog manager: success
I20260619 06:56:19.808854 8769 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f5575c44f96542caa79844a15fbcfce6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:56:19.913466 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.907174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.18.39.125:43789)
0619 06:56:19.907357 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.907362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.907703 (+ 341us) server_negotiation.cc:407] Connection header received
0619 06:56:19.907888 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.907892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.907973 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.908062 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.908469 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.909214 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.911127 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.911853 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.912038 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.912718 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.912848 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.912933 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.913050 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.913118 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:56:19.924649 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.919110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.18.39.124:34601)
0619 06:56:19.919323 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.919328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.919537 (+ 209us) server_negotiation.cc:407] Connection header received
0619 06:56:19.919672 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.919676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.919830 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.919931 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.920263 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.920975 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.922330 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.922761 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.922871 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.923356 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.923955 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.924084 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.924157 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.924226 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:56:19.939023 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.932418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.18.39.125:43789)
0619 06:56:19.932599 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.932606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.932911 (+ 305us) server_negotiation.cc:407] Connection header received
0619 06:56:19.933125 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.933130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.933229 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.933305 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.934142 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.934974 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.937115 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.937748 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.937899 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.938279 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.938380 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.938477 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.938620 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.938702 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:56:19.939420 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.931375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48308 (local address 127.18.39.126:39333)
0619 06:56:19.931870 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.931878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.932251 (+ 373us) server_negotiation.cc:407] Connection header received
0619 06:56:19.932613 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.932619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.932920 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.933091 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.933994 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.934994 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.936808 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.937373 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.937488 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.938022 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.938515 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.938614 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.938995 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.939089 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260619 06:56:19.939644 8754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:19.931556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.18.39.124:34601)
0619 06:56:19.931752 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:19.931758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:19.933497 (+ 1739us) server_negotiation.cc:407] Connection header received
0619 06:56:19.933718 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:19.933723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:19.933796 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:19.933892 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:19.934490 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.935546 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.937577 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:19.938182 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:19.938269 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:19.938897 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:19.939020 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:19.939082 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:19.939288 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 06:56:19.939363 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:56:19.944211 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8536
I20260619 06:56:19.945657 8598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:20.059890 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8536
W20260619 06:56:20.080662 8626 connection.cc:535] server connection from 127.0.0.1:33634 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 06:56:20.081045 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8610
I20260619 06:56:20.082211 8672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:20.191767 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8610
I20260619 06:56:20.212198 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8684
I20260619 06:56:20.213428 8746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:20.322350 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8684
2026-06-19T06:56:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4507 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:56:20 dist-test-slave-1h5s krb5kdc[8931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:56:20 dist-test-slave-1h5s krb5kdc[8931](info): set up 2 sockets
Jun 19 06:56:20 dist-test-slave-1h5s krb5kdc[8931](info): commencing operation
krb5kdc: starting...
W20260619 06:56:21.446635 18589 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/test-user.keytab.
Jun 19 06:56:21 dist-test-slave-1h5s krb5kdc[8931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:56:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.39.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
Entry for principal kudu/127.18.39.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab.
I20260619 06:56:21.718760 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34313
--webserver_interface=127.18.39.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:56:22.059600 8944 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:22.060063 8944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:22.060194 8944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:22.060334 8944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:22.067829 8944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:22.068166 8944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:22.068360 8944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:22.068459 8944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:22.068557 8944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:22.079177 8944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.126.keytab
--principal=kudu/127.18.39.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.126:34313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.39.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:22.080657 8944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:22.082048 8944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:22.093916 8949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:22.093919 8950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:22.093962 8952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:22.095290 8944 server_base.cc:1034] running on GCE node
I20260619 06:56:22.096165 8944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:22.097858 8944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:22.099123 8944 hybrid_clock.cc:648] HybridClock initialized: now 1781852182099061 us; error 43 us; skew 500 ppm
I20260619 06:56:22.099524 8944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:22.102859 8944 init.cc:377] Logged in from keytab as kudu/127.18.39.126@KRBTEST.COM (short username kudu)
I20260619 06:56:22.104576 8944 webserver.cc:460] Webserver started at http://127.18.39.126:42213/ using document root <none> and password file <none>
I20260619 06:56:22.105387 8944 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:22.105526 8944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:22.105854 8944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:22.108944 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data/instance:
uuid: "9d863f80967b48708630365877d4db36"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "d781e4657b9ed7a9d5b5e86e574f681a"
server_key_iv: "fdc6fb2caadbfde8cb537fd838717105"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.109813 8944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance:
uuid: "9d863f80967b48708630365877d4db36"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "d781e4657b9ed7a9d5b5e86e574f681a"
server_key_iv: "fdc6fb2caadbfde8cb537fd838717105"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.115276 8944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:56:22.119127 8959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:22.120515 8944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:56:22.120723 8944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal
uuid: "9d863f80967b48708630365877d4db36"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "d781e4657b9ed7a9d5b5e86e574f681a"
server_key_iv: "fdc6fb2caadbfde8cb537fd838717105"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.120966 8944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:22.145299 8944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:22.149847 8944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:22.150161 8944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:22.169641 8944 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.126:34313
I20260619 06:56:22.169659 9010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.126:34313 every 8 connection(s)
I20260619 06:56:22.171190 8944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/data/info.pb
I20260619 06:56:22.175298 9011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:22.176177 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 8944
I20260619 06:56:22.176366 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-0/wal/instance
I20260619 06:56:22.176831 18589 external_mini_cluster.cc:1428] Setting key fdabce4f51b4fd83ff9fc2447d654230
I20260619 06:56:22.182942 9011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.125@KRBTEST.COM" created.
I20260619 06:56:22.209529 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.186595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34313 (local address 127.0.0.1:41042)
0619 06:56:22.187724 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.187788 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:22.189958 (+ 2170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:22.191047 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:22.191052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:22.191699 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:22.192076 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.192098 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.194694 (+ 2596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.194699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.196014 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.196032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.197654 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.197659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.197831 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.198583 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:22.198608 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:22.201049 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:22.204848 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.204861 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.204902 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.205481 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.205936 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.205941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.205944 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.206102 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.206920 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:22.206929 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:22.207136 (+ 207us) client_negotiation.cc:769] Sending connection context
0619 06:56:22.207390 (+ 254us) client_negotiation.cc:240] Negotiation successful
0619 06:56:22.208060 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":161,"thread_start_us":462,"threads_started":1}
I20260619 06:56:22.209550 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.186587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.18.39.126:34313)
0619 06:56:22.188414 (+ 1827us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:22.188424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:22.188480 (+ 56us) server_negotiation.cc:407] Connection header received
0619 06:56:22.190401 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:22.190413 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:22.190771 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:22.191012 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:22.192326 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.194465 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.196207 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.197368 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.197589 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.201385 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:22.201402 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:22.201423 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:22.201477 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:22.204645 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.205625 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.205642 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.205644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.205802 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.206207 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.206215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.206218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.206768 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:22.206964 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:22.207515 (+ 551us) server_negotiation.cc:299] Negotiation successful
0619 06:56:22.207936 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":1606,"spinlock_wait_cycles":56576,"thread_start_us":1377,"threads_started":1}
I20260619 06:56:22.212896 9011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:22.214174 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.213595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51154)
0619 06:56:22.213811 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.214003 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 06:56:22.214465 8961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
I20260619 06:56:22.216223 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.215725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51160)
0619 06:56:22.215927 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.216026 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 06:56:22.216435 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:42203: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
Entry for principal kudu/127.18.39.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab.
I20260619 06:56:22.236260 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:42203
--webserver_interface=127.18.39.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:56:22.265818 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } attempt: 1
I20260619 06:56:22.267135 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.266580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51170)
0619 06:56:22.266820 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.266969 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 06:56:22.268201 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.267800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51180)
0619 06:56:22.267967 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.268056 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 06:56:22.268411 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:42203: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
I20260619 06:56:22.336694 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } attempt: 2
I20260619 06:56:22.338104 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.337498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51182)
0619 06:56:22.337815 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.337934 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 06:56:22.339248 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.338797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51198)
0619 06:56:22.338998 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.339083 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260619 06:56:22.339442 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:42203: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
I20260619 06:56:22.430721 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } attempt: 3
I20260619 06:56:22.431989 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.431507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51204)
0619 06:56:22.431721 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.431819 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:56:22.433226 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.432805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51206)
0619 06:56:22.432980 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.433067 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:56:22.433562 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:42203: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
I20260619 06:56:22.576810 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } attempt: 4
I20260619 06:56:22.578133 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.577588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51216)
0619 06:56:22.577910 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.578001 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260619 06:56:22.579145 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.578780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51220)
0619 06:56:22.578929 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.579026 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:56:22.579329 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.125:42203: Network error: Client connection negotiation failed: client connection to 127.18.39.125:42203: connect: Connection refused (error 111)
W20260619 06:56:22.579334 9018 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:22.579751 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:22.579869 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:22.580020 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:22.587634 9018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:22.587783 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:22.587886 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:22.587991 9018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:22.588080 9018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:22.598668 9018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.125.keytab
--principal=kudu/127.18.39.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.125:42203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.39.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:22.600147 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:22.601588 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:22.614941 9026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:22.615000 9024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:22.614977 9023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:22.615594 9018 server_base.cc:1034] running on GCE node
I20260619 06:56:22.616467 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:22.617987 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:22.619228 9018 hybrid_clock.cc:648] HybridClock initialized: now 1781852182619140 us; error 66 us; skew 500 ppm
I20260619 06:56:22.619575 9018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:22.623050 9018 init.cc:377] Logged in from keytab as kudu/127.18.39.125@KRBTEST.COM (short username kudu)
I20260619 06:56:22.624819 9018 webserver.cc:460] Webserver started at http://127.18.39.125:37949/ using document root <none> and password file <none>
I20260619 06:56:22.625720 9018 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:22.625833 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:22.626187 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:22.629673 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data/instance:
uuid: "3efd0e0a9f1a471d8339484423282183"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "fac9d22655554c20282d2036ecc6ba08"
server_key_iv: "c8de2941673536ec61f7d6a8b4481f62"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.630548 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal/instance:
uuid: "3efd0e0a9f1a471d8339484423282183"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "fac9d22655554c20282d2036ecc6ba08"
server_key_iv: "c8de2941673536ec61f7d6a8b4481f62"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.636359 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:56:22.640089 9033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:22.641580 9018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:56:22.641738 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal
uuid: "3efd0e0a9f1a471d8339484423282183"
format_stamp: "Formatted at 2026-06-19 06:56:22 on dist-test-slave-1h5s"
server_key: "fac9d22655554c20282d2036ecc6ba08"
server_key_iv: "c8de2941673536ec61f7d6a8b4481f62"
server_key_version: "encryptionkey@0"
I20260619 06:56:22.641961 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:22.659495 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:22.664481 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:22.664811 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:22.684725 9018 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.125:42203
I20260619 06:56:22.684733 9084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.125:42203 every 8 connection(s)
I20260619 06:56:22.686321 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/data/info.pb
I20260619 06:56:22.690212 9085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:22.693758 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 9018
I20260619 06:56:22.693949 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-1/wal/instance
I20260619 06:56:22.694540 18589 external_mini_cluster.cc:1428] Setting key d0e3f80c7f7f660a02070a1cc6ec9022
I20260619 06:56:22.699189 9085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.39.124@KRBTEST.COM; defaulting to no policy
I20260619 06:56:22.725425 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.702944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.18.39.126:34313)
0619 06:56:22.703189 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:22.703199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:22.704411 (+ 1212us) server_negotiation.cc:407] Connection header received
0619 06:56:22.706997 (+ 2586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:22.707002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:22.707077 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:22.707174 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:22.709387 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.710100 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.712561 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.713026 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.713133 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.717800 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:22.717812 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:22.717820 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:22.717874 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:22.721793 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.722747 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.722754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.722758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.722937 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.723379 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.723385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.723391 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.723807 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:22.723997 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:22.724783 (+ 786us) server_negotiation.cc:299] Negotiation successful
0619 06:56:22.725043 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.39.124@KRBTEST.COM" created.
I20260619 06:56:22.726639 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.703349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34313 (local address 127.0.0.1:41054)
0619 06:56:22.704219 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.704267 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:22.706671 (+ 2404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:22.707445 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:22.707460 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:22.708294 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:22.709125 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.709151 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.710270 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.710278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.712373 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.712393 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.713274 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.713283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.713651 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.714461 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:22.714495 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:22.717500 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:22.721998 (+ 4498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.722006 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.722022 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.722594 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.723068 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.723071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.723073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.723253 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.723987 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:22.723994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:22.724354 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 06:56:22.724628 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 06:56:22.725132 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":392,"threads_started":1}
I20260619 06:56:22.728495 9085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:22.742767 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.729132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51224)
0619 06:56:22.729301 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.729370 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:22.729475 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:22.730473 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:22.730478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:22.730931 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:22.731061 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.731068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.732402 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.732407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.733295 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.733306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.734206 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.734212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.734383 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.735004 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:22.735022 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:22.736989 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:22.740222 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.740230 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.740236 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.740596 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.740954 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.740960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.740963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.741098 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.741750 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:22.741755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:22.741888 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 06:56:22.742042 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 06:56:22.742354 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:56:22.742878 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.729464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.18.39.125:42203)
0619 06:56:22.730067 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:22.730073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:22.730098 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:56:22.730144 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:22.730149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:22.730327 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:22.730497 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:22.731231 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.732262 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.733492 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.734043 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.734188 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.737201 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:22.737213 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:22.737228 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:22.737255 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:22.740085 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.740725 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.740739 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.740741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.740842 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.741206 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.741211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.741214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.741617 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:22.741824 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:22.742202 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 06:56:22.742479 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":462,"thread_start_us":329,"threads_started":1}
I20260619 06:56:22.745687 9085 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:22.747099 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.746432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38800)
0619 06:56:22.746697 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.746866 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260619 06:56:22.747387 9034 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:22.749125 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.748635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38816)
0619 06:56:22.748824 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.748935 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 06:56:22.749439 9085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
Entry for principal kudu/127.18.39.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab.
I20260619 06:56:22.758176 18589 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
/tmp/dist-test-taskHLTYaR/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:45903
--webserver_interface=127.18.39.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.39.84:34125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:56:22.798827 9085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 1
I20260619 06:56:22.800101 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.799542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38830)
0619 06:56:22.799767 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.799909 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:56:22.801157 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.800746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38844)
0619 06:56:22.800924 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.801016 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 06:56:22.801416 9085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:22.869678 9085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 2
I20260619 06:56:22.871076 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.870479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38852)
0619 06:56:22.870749 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.870879 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 06:56:22.872440 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.871921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38856)
0619 06:56:22.872130 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.872249 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 06:56:22.872716 9085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:22.878608 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } attempt: 5
Jun 19 06:56:22 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:22.892457 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.879554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.18.39.125:42203)
0619 06:56:22.879749 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:22.879755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:22.879778 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:56:22.879852 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:22.879857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:22.879937 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:22.880047 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:22.880843 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.881632 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.882811 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.883243 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.883383 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.886774 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:22.886780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:22.886783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:22.886815 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:22.889782 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.890510 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.890516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.890521 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.890648 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:22.891033 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:22.891038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:22.891042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:22.891334 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:22.891495 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:22.891897 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 06:56:22.892120 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:56:22.892478 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.879389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51236)
0619 06:56:22.879580 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.879605 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:22.879745 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:22.880077 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:22.880082 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:22.880504 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:22.880686 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.880696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.881769 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.881776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.882648 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:22.882668 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:22.883389 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:22.883395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:22.883628 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:22.884275 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:22.884292 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:22.886561 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:22.889950 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.889957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.889961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.890380 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.890764 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:22.890769 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:22.890773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:22.890924 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:22.891457 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:22.891462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:22.891634 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 06:56:22.891779 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 06:56:22.892102 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 06:56:22.893775 9011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:56:22.895118 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.894623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38866)
0619 06:56:22.894843 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.894965 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 06:56:22.896245 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.895869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38872)
0619 06:56:22.896063 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.896143 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260619 06:56:22.896518 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:22.947755 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 1
I20260619 06:56:22.949050 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.948565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38878)
0619 06:56:22.948770 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.948884 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:56:22.950131 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.949814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38886)
0619 06:56:22.949967 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.950034 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260619 06:56:22.950366 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:22.964001 9085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 3
I20260619 06:56:22.965445 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.964729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38888)
0619 06:56:22.964992 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.965143 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:56:22.966909 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:22.966273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38904)
0619 06:56:22.966591 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:22.966719 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260619 06:56:22.967172 9085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:23.018621 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 2
I20260619 06:56:23.019896 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.019383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38920)
0619 06:56:23.019625 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.019720 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 06:56:23.020936 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.020583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38922)
0619 06:56:23.020730 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.020831 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:56:23.021173 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:23.110440 9085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 4
I20260619 06:56:23.111660 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.111113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38932)
0619 06:56:23.111353 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.111468 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 06:56:23.112679 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.112328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38946)
0619 06:56:23.112475 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.112553 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 06:56:23.112910 9085 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
W20260619 06:56:23.113891 9092 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:56:23.114277 9092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:56:23.114393 9092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:56:23.114548 9092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:56:23.122128 9092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:56:23.122269 9092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:56:23.122377 9092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:56:23.122470 9092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:56:23.122558 9092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:56:23.127417 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 3
I20260619 06:56:23.128535 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.128049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38962)
0619 06:56:23.128296 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.128383 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 06:56:23.129535 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.129163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38964)
0619 06:56:23.129363 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.129439 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260619 06:56:23.129732 9011 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.39.124:45903: Network error: Client connection negotiation failed: client connection to 127.18.39.124:45903: connect: Connection refused (error 111)
I20260619 06:56:23.133189 9092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.39.84:34125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.39.126:34313,127.18.39.125:42203,127.18.39.124:45903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/krb5kdc/kudu_127.18.39.124.keytab
--principal=kudu/127.18.39.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.39.124:45903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.39.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:56:23.134637 9092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:56:23.136435 9092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:56:23.148970 9097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:23.149000 9100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:56:23.149051 9098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:56:23.150148 9092 server_base.cc:1034] running on GCE node
I20260619 06:56:23.151012 9092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:56:23.152534 9092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:56:23.153779 9092 hybrid_clock.cc:648] HybridClock initialized: now 1781852183153674 us; error 113 us; skew 500 ppm
I20260619 06:56:23.154174 9092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852183, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:56:23.157426 9092 init.cc:377] Logged in from keytab as kudu/127.18.39.124@KRBTEST.COM (short username kudu)
I20260619 06:56:23.158986 9092 webserver.cc:460] Webserver started at http://127.18.39.124:33213/ using document root <none> and password file <none>
I20260619 06:56:23.159677 9092 fs_manager.cc:359] Metadata directory not provided
I20260619 06:56:23.159768 9092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:56:23.160055 9092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:56:23.163161 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data/instance:
uuid: "59b4360437424ddab5abef866083e8f3"
format_stamp: "Formatted at 2026-06-19 06:56:23 on dist-test-slave-1h5s"
server_key: "fd73387bc9fe44ca94f648bdf893ff79"
server_key_iv: "ba17b00765cdce936f8e38708806e5a8"
server_key_version: "encryptionkey@0"
I20260619 06:56:23.163929 9092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal/instance:
uuid: "59b4360437424ddab5abef866083e8f3"
format_stamp: "Formatted at 2026-06-19 06:56:23 on dist-test-slave-1h5s"
server_key: "fd73387bc9fe44ca94f648bdf893ff79"
server_key_iv: "ba17b00765cdce936f8e38708806e5a8"
server_key_version: "encryptionkey@0"
I20260619 06:56:23.169780 9092 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 06:56:23.174705 9107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:56:23.176474 9092 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 06:56:23.176684 9092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data,/tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal
uuid: "59b4360437424ddab5abef866083e8f3"
format_stamp: "Formatted at 2026-06-19 06:56:23 on dist-test-slave-1h5s"
server_key: "fd73387bc9fe44ca94f648bdf893ff79"
server_key_iv: "ba17b00765cdce936f8e38708806e5a8"
server_key_version: "encryptionkey@0"
I20260619 06:56:23.176978 9092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:56:23.195380 9092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:56:23.200019 9092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:56:23.200266 9092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:56:23.219754 9092 rpc_server.cc:307] RPC server started. Bound to: 127.18.39.124:45903
I20260619 06:56:23.219871 9158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.39.124:45903 every 8 connection(s)
I20260619 06:56:23.221258 9092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/data/info.pb
I20260619 06:56:23.225289 9159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:56:23.225817 18589 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu as pid 9092
I20260619 06:56:23.226008 18589 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskHLTYaR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851818886250-18589-0/minicluster-data/master-2/wal/instance
I20260619 06:56:23.226455 18589 external_mini_cluster.cc:1428] Setting key d7591251e3d46ee0bedc6297d2b9d553
I20260619 06:56:23.234632 9159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852183, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
I20260619 06:56:23.251572 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.228202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.18.39.124:45903)
0619 06:56:23.229019 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.229031 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.229073 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:56:23.231473 (+ 2400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.231498 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.231946 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.232290 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.234082 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.235455 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.237253 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.238547 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.238790 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.243414 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.243426 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.243442 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.243497 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.247149 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.247950 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.247959 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.247975 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.248126 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.248533 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.248540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.248544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.249226 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.249580 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.250052 (+ 472us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.250505 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":396,"threads_started":1}
I20260619 06:56:23.253939 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.237593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.126:34313 (local address 127.0.0.1:41056)
0619 06:56:23.238462 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.238496 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:23.238678 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:23.239153 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:23.239159 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:23.240088 (+ 929us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:23.240443 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.240457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.241601 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.241610 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.243039 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.243055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.244220 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.244227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.244469 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.245197 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:23.245223 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:23.247830 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:23.251187 (+ 3357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.251200 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.251208 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.251683 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.252154 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.252159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.252163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.252290 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.252898 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:23.252908 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:23.253093 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 06:56:23.253261 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 06:56:23.253583 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":528,"threads_started":1}
I20260619 06:56:23.254329 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.237526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.18.39.126:34313)
0619 06:56:23.238079 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.238087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.238593 (+ 506us) server_negotiation.cc:407] Connection header received
0619 06:56:23.238840 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.238846 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.238942 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.239045 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.240617 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.241448 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.243248 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.243981 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.244150 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.248064 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.248070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.248074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.248108 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.251056 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.251862 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.251865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.251903 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.252030 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.252416 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.252423 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.252426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.252738 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.252926 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.253429 (+ 503us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.253660 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":273,"threads_started":1}
I20260619 06:56:23.255880 9159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.126@KRBTEST.COM
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852183, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:23.270771 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.255066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.18.39.126:34313)
0619 06:56:23.255225 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.255232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.255274 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:56:23.255458 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.255461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.255529 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.255603 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.257066 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.258145 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.259862 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.260479 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.260609 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.265046 (+ 4437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.265053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.265058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.265102 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.268016 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.268745 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.268750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.268752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.268863 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.269259 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.269266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.269271 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.269635 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.269918 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.270206 (+ 288us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.270429 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:56:23.270771 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.256564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.125:42203 (local address 127.0.0.1:51250)
0619 06:56:23.256751 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.256771 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:23.256914 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:23.257301 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:23.257305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:23.257799 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:23.257983 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.257994 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.259402 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.259405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.260784 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.260798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.261718 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.261722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.261852 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.262417 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:23.262435 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:23.265057 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:23.268173 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.268179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.268183 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.268584 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.269054 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.269057 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.269060 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.269190 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.269787 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:23.269792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:23.269975 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 06:56:23.270135 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 06:56:23.270431 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 06:56:23.271054 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.256809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.18.39.125:42203)
0619 06:56:23.257028 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.257036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.257055 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:56:23.257117 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.257120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.257177 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.257248 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.258133 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.259211 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.260945 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.261562 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.261703 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.265267 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.265271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.265275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.265314 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.268016 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.268710 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.268717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.268722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.268910 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.269325 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.269328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.269370 (+ 42us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.269645 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.269810 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.270256 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.270475 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:56:23.271965 9159 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852183, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.124@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.39.125@KRBTEST.COM
I20260619 06:56:23.285681 9164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.272680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38988)
0619 06:56:23.272977 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.273000 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:23.273077 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:23.273389 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:23.273394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:23.273791 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:23.273917 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.273925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.274948 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.274954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.275791 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.275801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.276459 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.276464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.276673 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.277275 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:23.277293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:23.279649 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:23.283073 (+ 3424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.283079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.283082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.283567 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.283941 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.283949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.283955 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.284130 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.284792 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:23.284796 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:23.284942 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 06:56:23.285085 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 06:56:23.285399 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260619 06:56:23.286000 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.272856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38988 (local address 127.18.39.124:45903)
0619 06:56:23.273031 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.273035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.273048 (+ 13us) server_negotiation.cc:407] Connection header received
0619 06:56:23.273187 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.273192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.273251 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.273376 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.274044 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.274811 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.275925 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.276303 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.276440 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.279849 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.279855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.279860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.279899 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.282940 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.283688 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.283692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.283694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.283847 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.284241 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.284247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.284250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.284666 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.284824 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.285212 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.285569 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":62}
I20260619 06:56:23.288576 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.272734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51254 (local address 127.18.39.125:42203)
0619 06:56:23.272906 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.272914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.272932 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:56:23.273247 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.273252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.273325 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.273452 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.274763 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.275485 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.276626 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.277065 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.277202 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.282219 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.282224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.282228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.282261 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.285741 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.286526 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.286532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.286536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.286680 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.287063 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.287068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.287072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.287454 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.287734 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.287991 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.288233 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:56:23.291970 9159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: Bootstrap starting.
I20260619 06:56:23.293747 18589 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:56:23.295724 9159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:23.296922 9159 log.cc:824] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:23.296343 8986 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9d863f80967b48708630365877d4db36"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41060
I20260619 06:56:23.300031 9159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: No bootstrap required, opened a new log
I20260619 06:56:23.306972 9011 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 4
I20260619 06:56:23.307695 9159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.308205 9159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b4360437424ddab5abef866083e8f3, State: Initialized, Role: FOLLOWER
I20260619 06:56:23.308807 9159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.310029 9166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.310429 9166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.311440 9159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.126@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:56:23.326933 9014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.307773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:38998)
0619 06:56:23.307987 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.308011 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:23.308146 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:23.308477 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:23.308481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:23.309004 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:23.309216 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.309232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.310782 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.310787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.312218 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.312234 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.313566 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.313576 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.313831 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.314607 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:23.314631 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:23.317745 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:23.323169 (+ 5424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.323178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.323183 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.323867 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.324463 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.324472 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.324478 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.324700 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.325797 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:23.325802 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:23.326014 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 06:56:23.326180 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 06:56:23.326526 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 06:56:23.326967 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.307842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.18.39.124:45903)
0619 06:56:23.308098 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.308104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.308176 (+ 72us) server_negotiation.cc:407] Connection header received
0619 06:56:23.308249 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.308254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.308334 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.308444 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.309396 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.310615 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.312409 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.313280 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.313529 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.318013 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.318021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.318025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.318064 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.322942 (+ 4878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.324033 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.324041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.324050 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.324279 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.324916 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.324924 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.324930 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.325639 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.325827 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.326313 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.326559 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 06:56:23.319787 9092 master_runner.cc:186] Error getting master registration for 127.18.39.126:34313: OK, instance_id { permanent_uuid: "9d863f80967b48708630365877d4db36" instance_seqno: 1781852182164246 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:56:23.332195 9092 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:56:23.334009 9011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: Bootstrap starting.
W20260619 06:56:23.335728 9177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:56:23.335867 9177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:56:23.338133 9011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:23.339465 9011 log.cc:824] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:23.342864 9011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: No bootstrap required, opened a new log
I20260619 06:56:23.350550 9011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.351018 9011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d863f80967b48708630365877d4db36, State: Initialized, Role: FOLLOWER
I20260619 06:56:23.351585 9011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.352669 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.353152 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.354019 9011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:23.354176 8986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 06:56:23.354415 8986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:23.358189 8986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.360967 9134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d863f80967b48708630365877d4db36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "59b4360437424ddab5abef866083e8f3"
I20260619 06:56:23.361542 9134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:23.365201 9134 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d863f80967b48708630365877d4db36 in term 1.
I20260619 06:56:23.365711 8986 leader_election.cc:288] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [CANDIDATE]: Term 1 election: Requested vote from peers 3efd0e0a9f1a471d8339484423282183 (127.18.39.125:42203), 59b4360437424ddab5abef866083e8f3 (127.18.39.124:45903)
I20260619 06:56:23.366102 8960 leader_election.cc:302] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b4360437424ddab5abef866083e8f3, 9d863f80967b48708630365877d4db36; no voters:
I20260619 06:56:23.366766 9180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:56:23.368537 9180 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [term 1 LEADER]: Becoming Leader. State: Replica: 9d863f80967b48708630365877d4db36, State: Running, Role: LEADER
I20260619 06:56:23.369082 9180 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.379824 9191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d863f80967b48708630365877d4db36. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.380330 9191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:23.381781 9198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:56:23.399968 9198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:56:23.412127 9085 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } attempt: 5
I20260619 06:56:23.416921 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.373069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39010 (local address 127.18.39.124:45903)
0619 06:56:23.373232 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.373238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.388102 (+ 14864us) server_negotiation.cc:407] Connection header received
0619 06:56:23.388438 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.388448 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.388584 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.388694 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.396208 (+ 7514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.397433 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.401051 (+ 3618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.401822 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.401933 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.406953 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.406964 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.406971 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.407031 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.411803 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.413541 (+ 1738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.413552 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.413558 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.413790 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.414400 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.414408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.414414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.414897 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.415242 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.416038 (+ 796us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.416369 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 06:56:23.423645 8965 tablet.cc:2369] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 06:56:23 dist-test-slave-1h5s krb5kdc[8931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781852182, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.39.125@KRBTEST.COM for kudu/127.18.39.124@KRBTEST.COM
I20260619 06:56:23.442312 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.387000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.18.39.125:42203)
0619 06:56:23.387250 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.387259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.403293 (+ 16034us) server_negotiation.cc:407] Connection header received
0619 06:56:23.420939 (+ 17646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.420946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.421030 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.421098 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.423650 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.424791 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.429448 (+ 4657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.430375 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.430515 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.434552 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.434560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.434568 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.434609 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.438138 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.439024 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.439030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.439034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.439201 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.439704 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.439708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.439712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.440128 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.440401 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.441122 (+ 721us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.441704 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:56:23.443621 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.412844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.39.124:45903 (local address 127.0.0.1:39024)
0619 06:56:23.413280 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 06:56:23.413303 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:56:23.413446 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:56:23.417894 (+ 4448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:56:23.417903 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:56:23.418567 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:56:23.418888 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.418905 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.422394 (+ 3489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.422400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.423870 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:56:23.423886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.425017 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.425026 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:56:23.425477 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.426329 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:56:23.426354 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:56:23.429263 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:56:23.439894 (+ 10631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.439899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.439903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.440427 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.441785 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:56:23.441789 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:56:23.441792 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:56:23.441963 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:56:23.442651 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:56:23.442656 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:56:23.442806 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 06:56:23.442954 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 06:56:23.443251 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260619 06:56:23.444739 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.413828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39024 (local address 127.18.39.124:45903)
0619 06:56:23.417468 (+ 3640us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.417476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.417509 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:56:23.417571 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.417578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.417675 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.419061 (+ 1386us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.419194 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.422221 (+ 3027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.424075 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.424758 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.424951 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.432065 (+ 7114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.432072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.432077 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.432125 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.439736 (+ 7611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.441454 (+ 1718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.441459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.441462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.441645 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.442122 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.442127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.442130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.442516 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.443866 (+ 1350us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.444018 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.444279 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3345,"thread_start_us":24009,"threads_started":1}
I20260619 06:56:23.401254 8944 master_runner.cc:186] Error getting master registration for 127.18.39.125:42203: OK, instance_id { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" instance_seqno: 1781852182679000 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:56:23.449927 8944 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:56:23.451941 9085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: Bootstrap starting.
W20260619 06:56:23.452441 9113 tablet.cc:2369] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:56:23.453079 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.372646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.18.39.126:34313)
0619 06:56:23.372843 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.372850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.387534 (+ 14684us) server_negotiation.cc:407] Connection header received
0619 06:56:23.388510 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.388518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.388611 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.388697 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.403234 (+ 14537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.420009 (+ 16775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.422106 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.425182 (+ 3076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.425325 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.432962 (+ 7637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:56:23.432969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:56:23.432974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:56:23.433020 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:56:23.442379 (+ 9359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.445017 (+ 2638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.445024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.445028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.445219 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:56:23.449536 (+ 4317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:56:23.449543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:56:23.449546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:56:23.450032 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:56:23.450359 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.452400 (+ 2041us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.452695 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 06:56:23.458101 9085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: Neither blocks nor log segments found. Creating new log.
I20260619 06:56:23.460381 9085 log.cc:824] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: Log is configured to *not* fsync() on all Append() calls
I20260619 06:56:23.466955 9134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [term 1 FOLLOWER]: Refusing update from remote peer 9d863f80967b48708630365877d4db36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:56:23.468708 9180 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:56:23.492884 9166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d863f80967b48708630365877d4db36. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.493255 9166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.493418 9204 mvcc.cc:205] Tried to move back new op lower bound from 7298466543468343296 to 7298466543085686784. Current Snapshot: MvccSnapshot[applied={T|T < 7298466543468343296}]
I20260619 06:56:23.508737 9085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: No bootstrap required, opened a new log
I20260619 06:56:23.515137 9166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.515555 9166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.526403 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.526736 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:23.535179 9198 catalog_manager.cc:1347] Generated new cluster ID: c556b57aa05b408f8590b25bc1b582c1
I20260619 06:56:23.535311 9198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:56:23.537856 9085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.541571 9085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efd0e0a9f1a471d8339484423282183, State: Initialized, Role: FOLLOWER
I20260619 06:56:23.542534 9085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } }
I20260619 06:56:23.557967 9210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.559695 9210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.560933 9060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d863f80967b48708630365877d4db36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3efd0e0a9f1a471d8339484423282183"
I20260619 06:56:23.561321 9059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:56:23.561507 9085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:56:23.565894 9059 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [term 1 FOLLOWER]: Refusing update from remote peer 9d863f80967b48708630365877d4db36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:56:23.567157 9191 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:56:23.575292 9198 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:56:23.573064 9060 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d863f80967b48708630365877d4db36 in term 1.
I20260619 06:56:23.577531 9198 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:56:23.622960 9198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36: Generated new TSK 0
I20260619 06:56:23.630393 9198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:56:23.631634 9210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d863f80967b48708630365877d4db36. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.631676 9215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.632047 9210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.632048 9215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:56:23.638511 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d863f80967b48708630365877d4db36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d863f80967b48708630365877d4db36" member_type: VOTER last_known_addr { host: "127.18.39.126" port: 34313 } } peers { permanent_uuid: "3efd0e0a9f1a471d8339484423282183" member_type: VOTER last_known_addr { host: "127.18.39.125" port: 42203 } } peers { permanent_uuid: "59b4360437424ddab5abef866083e8f3" member_type: VOTER last_known_addr { host: "127.18.39.124" port: 45903 } } }
I20260619 06:56:23.638913 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d863f80967b48708630365877d4db36 [sys.catalog]: This master's current role is: LEADER
I20260619 06:56:23.641897 9213 mvcc.cc:205] Tried to move back new op lower bound from 7298466543468343296 to 7298466543085686784. Current Snapshot: MvccSnapshot[applied={T|T < 7298466543468343296}]
W20260619 06:56:23.752908 18589 mini_kdc.cc:348] Time spent kdestroy: real 0.101s user 0.000s sys 0.026s
I20260619 06:56:23.778370 9228 catalog_manager.cc:1259] Loaded cluster ID: c556b57aa05b408f8590b25bc1b582c1
I20260619 06:56:23.778510 9228 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: loading cluster ID for follower catalog manager: success
I20260619 06:56:23.786566 9228 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: acquiring CA information for follower catalog manager: success
I20260619 06:56:23.789467 9228 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3efd0e0a9f1a471d8339484423282183: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 06:56:23.816012 9163 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:23.807470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.18.39.126:34313)
0619 06:56:23.807704 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.807711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.813963 (+ 6252us) server_negotiation.cc:407] Connection header received
0619 06:56:23.814310 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.814317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.814415 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.814496 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.815743 (+ 1247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41080: BlockingRecv error: recv got EOF from 127.0.0.1:41080 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260619 06:56:23.819511 9089 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:23.814349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51272 (local address 127.18.39.125:42203)
0619 06:56:23.814573 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.814579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.816593 (+ 2014us) server_negotiation.cc:407] Connection header received
0619 06:56:23.818042 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.818054 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.818141 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.818219 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.819264 (+ 1045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51272: BlockingRecv error: recv got EOF from 127.0.0.1:51272 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260619 06:56:23.826143 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:23.815847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.18.39.124:45903)
0619 06:56:23.816049 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.816058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.821920 (+ 5862us) server_negotiation.cc:407] Connection header received
0619 06:56:23.824201 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.824208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.824327 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.824414 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.825879 (+ 1465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39032: BlockingRecv error: recv got EOF from 127.0.0.1:39032 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 06:56:23.873934 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.853674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.18.39.125:42203)
0619 06:56:23.853882 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.853888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.866060 (+ 12172us) server_negotiation.cc:407] Connection header received
0619 06:56:23.866376 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.866383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.866635 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.866847 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:23.867423 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.868630 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.870686 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.871372 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.871481 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.872263 (+ 782us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:23.873051 (+ 788us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:23.873170 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.873467 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.873585 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 06:56:23.921675 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:23.908387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.18.39.124:45903)
0619 06:56:23.908576 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.908583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.919365 (+ 10782us) server_negotiation.cc:407] Connection header received
0619 06:56:23.919935 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.919941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.920085 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.920164 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:23.921405 (+ 1241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39040: BlockingRecv error: recv got EOF from 127.0.0.1:39040 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:56:23.988265 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:23.964530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.18.39.125:42203)
0619 06:56:23.964797 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.964805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.981665 (+ 16860us) server_negotiation.cc:407] Connection header received
0619 06:56:23.982177 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.982183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:23.982281 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:23.982360 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:23.983051 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.984152 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.986124 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:23.986808 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:23.986985 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:23.987448 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:23.987615 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:23.987675 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:23.987894 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 06:56:23.987978 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 06:56:24.001274 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:23.999663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.18.39.124:45903)
0619 06:56:23.999896 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:23.999901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:23.999917 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:56:23.999983 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:23.999987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.000055 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.000273 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.001039 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39050: BlockingRecv error: recv got EOF from 127.0.0.1:39050 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 06:56:24.012305 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.006750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.18.39.125:42203)
0619 06:56:24.006967 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.006973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.007331 (+ 358us) server_negotiation.cc:407] Connection header received
0619 06:56:24.007578 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.007582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.007653 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.007764 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.008196 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.008971 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.010398 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.010816 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.010974 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.011394 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.011520 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.011655 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.011823 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.011923 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260619 06:56:24.021554 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.019226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.18.39.124:45903)
0619 06:56:24.019478 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.019488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.020064 (+ 576us) server_negotiation.cc:407] Connection header received
0619 06:56:24.020289 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.020295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.020387 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.020498 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.021264 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39066: BlockingRecv error: recv got EOF from 127.0.0.1:39066 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 06:56:24.034789 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.029051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51306 (local address 127.18.39.125:42203)
0619 06:56:24.029372 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.029381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.029922 (+ 541us) server_negotiation.cc:407] Connection header received
0619 06:56:24.030152 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.030158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.030242 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.030352 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.030771 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.031561 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.032933 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.033479 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.033617 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.034019 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.034141 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.034250 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.034396 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.034487 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 06:56:24.051708 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.041033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.18.39.124:45903)
0619 06:56:24.041424 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.041433 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.049963 (+ 8530us) server_negotiation.cc:407] Connection header received
0619 06:56:24.050203 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.050211 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.050316 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.050434 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.051462 (+ 1028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39068: BlockingRecv error: recv got EOF from 127.0.0.1:39068 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
I20260619 06:56:24.071313 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.064409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.18.39.125:42203)
0619 06:56:24.064598 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.064604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.065122 (+ 518us) server_negotiation.cc:407] Connection header received
0619 06:56:24.065359 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.065366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.065471 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.065568 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.066039 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.067208 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.069071 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.069842 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.069974 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.070455 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.070628 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.070723 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.070886 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.070975 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:56:24.081691 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.079379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.18.39.124:45903)
0619 06:56:24.079568 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.079575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.080088 (+ 513us) server_negotiation.cc:407] Connection header received
0619 06:56:24.080281 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.080286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.080384 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.080471 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.081302 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39084: BlockingRecv error: recv got EOF from 127.0.0.1:39084 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:56:24.107429 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.101370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.18.39.125:42203)
0619 06:56:24.101621 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.101629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.102009 (+ 380us) server_negotiation.cc:407] Connection header received
0619 06:56:24.102214 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.102219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.102293 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.102378 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.102828 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.103644 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.105556 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.106037 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.106155 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.106665 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.106774 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.106855 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.107057 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.107139 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 06:56:24.120102 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.114473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.18.39.124:45903)
0619 06:56:24.114725 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.114735 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.118628 (+ 3893us) server_negotiation.cc:407] Connection header received
0619 06:56:24.118856 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.118861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.118955 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.119065 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.119846 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39092: BlockingRecv error: recv got EOF from 127.0.0.1:39092 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:56:24.166900 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.159680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51348 (local address 127.18.39.125:42203)
0619 06:56:24.159926 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.159932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.160538 (+ 606us) server_negotiation.cc:407] Connection header received
0619 06:56:24.160751 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.160755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.160849 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.160958 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.161596 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.162783 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.164830 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.165433 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.165538 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.166049 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.166160 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.166227 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.166437 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.166512 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 06:56:24.175360 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.173175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39100 (local address 127.18.39.124:45903)
0619 06:56:24.173426 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.173435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.173966 (+ 531us) server_negotiation.cc:407] Connection header received
0619 06:56:24.174177 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.174189 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.174263 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.174369 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.175145 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39100: BlockingRecv error: recv got EOF from 127.0.0.1:39100 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:56:24.248701 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.242948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.18.39.125:42203)
0619 06:56:24.243170 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.243176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.243465 (+ 289us) server_negotiation.cc:407] Connection header received
0619 06:56:24.243650 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.243654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.243727 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.243808 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.244258 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.245022 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.246964 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.247440 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.247541 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.247996 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.248126 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.248191 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.248346 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.248411 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260619 06:56:24.256664 9162 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:56:24.254683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.18.39.124:45903)
0619 06:56:24.254884 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.254892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.255261 (+ 369us) server_negotiation.cc:407] Connection header received
0619 06:56:24.255444 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.255450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.255560 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.255674 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:56:24.256423 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39102: BlockingRecv error: recv got EOF from 127.0.0.1:39102 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:56:24.337971 9177 catalog_manager.cc:1259] Loaded cluster ID: c556b57aa05b408f8590b25bc1b582c1
I20260619 06:56:24.338083 9177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: loading cluster ID for follower catalog manager: success
I20260619 06:56:24.340569 9177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: acquiring CA information for follower catalog manager: success
I20260619 06:56:24.342424 9177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59b4360437424ddab5abef866083e8f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:56:24.393253 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.388158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.18.39.125:42203)
0619 06:56:24.388342 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.388346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.388675 (+ 329us) server_negotiation.cc:407] Connection header received
0619 06:56:24.388848 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.388852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.388927 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.389017 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.389508 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.390292 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.391596 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.392017 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.392116 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.392502 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.392624 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.392692 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.392907 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.392990 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:56:24.405409 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.399456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.18.39.124:45903)
0619 06:56:24.399698 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.399706 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.400033 (+ 327us) server_negotiation.cc:407] Connection header received
0619 06:56:24.400266 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.400270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.400431 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.400561 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.400966 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.401706 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.403107 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.403516 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.403657 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.404199 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.404747 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.404926 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.404982 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.405091 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:56:24.419242 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.411689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.18.39.124:45903)
0619 06:56:24.411947 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.411954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.412934 (+ 980us) server_negotiation.cc:407] Connection header received
0619 06:56:24.413175 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.413180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.413276 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.413396 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.414226 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.415271 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.417223 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.417865 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.418007 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.418465 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.418578 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.418675 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.418862 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.418950 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260619 06:56:24.421403 9089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.413496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.18.39.125:42203)
0619 06:56:24.413734 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.413740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.414541 (+ 801us) server_negotiation.cc:407] Connection header received
0619 06:56:24.414893 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.414900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.414996 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.415075 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.415824 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.416524 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.419084 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.419601 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.419721 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.420344 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.420469 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.420567 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.421008 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.421087 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:56:24.422240 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:56:24.411572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.18.39.126:34313)
0619 06:56:24.414154 (+ 2582us) server_negotiation.cc:206] Beginning negotiation
0619 06:56:24.414161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:56:24.414180 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:56:24.414315 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:56:24.414321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:56:24.414564 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:56:24.414747 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:56:24.415334 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.416476 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.419061 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:56:24.419962 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:56:24.420079 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:56:24.420765 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:56:24.421215 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:56:24.421362 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 06:56:24.421962 (+ 600us) server_negotiation.cc:299] Negotiation successful
0619 06:56:24.422031 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2442,"thread_start_us":2323,"threads_started":1}
I20260619 06:56:24.433596 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 8944
I20260619 06:56:24.434875 9006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:24.547122 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 8944
I20260619 06:56:24.568236 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 9018
I20260619 06:56:24.569406 9080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:24.679294 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 9018
I20260619 06:56:24.699640 18589 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu pid 9092
I20260619 06:56:24.700727 9154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:56:24.808589 18589 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskHLTYaR/build/asan/bin/kudu with pid 9092
2026-06-19T06:56:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4483 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8990 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (365886 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST